Document text
The Water Puri cation System for the
Daya Bay Reactor Neutrino Experiment
J. Wilhelmia, R. Boppc, R. Brownd, J. Cherwinkag, J. Cummingsb, E. Daled, M. Diwand, J.
Goettc, R.W. Hackenburgd, J. Kildu c, L. Littenbergd, G.S. Lif, X.N. Lie, J.C. Liue, H.Q.
Lue, J. Napolitanoa, C. Pearsond, N. Raperc, R. Roserod, P. Stolerc, Q. Xiaog, C.G. Yange,
Y. Yange, M. Yehd
aTemple University, Philadelphia, PA, USA
bSiena College, Loudonville, NY, USA
cRensselaer Polytechnic Institute, Troy, NY, USA
dBrookhaven National Laboratory, Upton, NY, USA
eInstitute of High Energy Physics, Beijing, China
fShanghai Jiao Tong University, Shanghai, China
gUniversity of Wisconsin, Madison, WI, USA
Abstract
We describe the design, installation, and operation of a puri cation system that is able
to provide large volumes of high purity ASTM (D1193-91) Type-I water to a high energy
physics experiment. The water environment is underground in a lightly sealed system, and
this provides signi cant challenges to maintaining high purity in the storage pools, each of
which contains several thousand cubic meters. High purity is dictated by the need for large
optical absorption length, which is critical for the operation of the experiment. The system
is largely successful, and the water clarity criteria are met. We also include a discussion of
lessons learned.
Keywords: High purity water, Deionization
1. Introduction
The Daya Bay Reactor Neutrino Experiment [1, 2, 3, 4] is a high energy particle physics
experiment which aims to measure properties of electron anti-neutrinos (or, for the rest of
this paper, simply \neutrinos") produced by the Daya Bay Nuclear Power Plant complex
located in Southeastern China. One major challenge to neutrino detection is their extremely
low probability of interaction requiring large detectors, in this case, gadolinium doped liq-
uid scintillator (Gd-LS) Antineutrino Detectors (ADs). Despite the high neutrino
ux in
Preprint submitted to Journal of Water Process Engineering August 7, 2014arXiv:1408.1302v1 [physics.ins-det] 6 Aug 2014
this experiment, there is major potential interference from a number of sources capable of
producing \non-neutrino" signals in the Gd-LS.
For example, the neutrino signal rate is 0:01 Hz in the detectors located nearest the
reactors, and0:001 Hz in the detectors farthest away. On the other hand, signals due
to radioactivity are 200 Hz (reduced from 100 kHz by the presence of the water pools
described in this paper), and the cosmic ray
ux is 1 Hz=m2in the near detector halls,
and0:04 Hz=m2in the far halls. Clearly these \background" sources need to be highly
suppressed.
To help solve these problems, we immerse the ADs in large pools of water, as part of a
comprehensive muon detection system [5]. As described below, this approach tackles back-
grounds both from natural radioactivity and penetrating cosmic rays. This paper focuses on
the relationship between water purity and the background suppression performance of the
pools.
Natural radioactivity sources include uranium and thorium series radioisotopes in the sur-
rounding rocks, as well as40K. Each AD has at least 2.5 m of water between it and the
walls providing sucient passive shielding against any rock-associated radioactivity. Radon,
a radioactive inert gas arising from U and Th series decay, can nd its way into the water
volume, and this leads to additional constraints on the design of the pools and the water
puri cation system.
The other important background, directly related to the clarity of water in the pools around
the ADs, is cosmic radiation. Energetic cosmic ray collisions in the upper atmosphere lead
to \showers" of muons and neutrons at the Earth's surface. The ADs are located under
hundreds of meters of rock to greatly reduce the
ux of cosmogenic neutrons. However,
muons are charged particles and produce Cherenkov light while passing through the water.
This light is monitored by arrays of 300 to 400 photomultiplier tubes (PMTs) providing a
veto signal to the ADs so that cosmic rays are not confused with neutrino events.
It is critical that the water be clear enough so that enough Cherenkov light reaches the PMTs.
The water pool PMTs are sensitive to light with wavelengths 300 nm to650 nm, with
a peak eciency at 400 nm, and Cherenkov light has a 1 =(wavelength)2dependence. The
DYB water pools are large, each ten meters deep, two of which measure 1016 m2and
the third measuring 1616 m2. Therefore, the experiment speci ed that the attenuation
length be more than 30 m for these wavelengths. This is well within the absorption length
of ultra pure water [6, 7], so our goal was to produce the highest purity water possible and
maintain it at as high a level as possible, within the constraints of this environment.
Figure 1 illustrates the challenge we faced, and the extent to which we met that challenge.
It plots (on a logarithmic vertical scale) the absorption coecient (i.e. inverse attenuation
length) as a function of wavelength for water under di erent circumstances. Transparency
measurements for ultra pure water [6, 7] are compared to our own analysis of samples of
source water and output from our primary puri cation stage. Our measurements were made
2
350 400 450 500 550 6000.0040.010.020.050.10.20.5
Wavelength (nm)Absorption Coefficient (1/m)
Source Water
Specification (1/30m)
EH4 Output
Sogandares and Fry
Pope and FryFigure 1: Absorption coecient (reciprocal of attenuation length) of water as a function of wavelength,
for DYB source water and for water treated by the Fill/MakeUp stage (EH4). We also show published
measurements [6, 7] of ultra high purity water, and indicate the 30 m attenuation length speci ed by the
neutrino physics experiment, in the wavelength range 400-500 nm.
with a Shimadzu spectrophotometer, model UV-1800, with a 10 cm long optical cell. An
analysis of the municipal source water indicates a turbidity of 0.31 NTU, a pH of 7.06, and
a conductivity (resitivity) of 88.4 S/cm (1:1310 2M
-cm). Although the source water
at Daya Bay, which comes from an open man-made reservoir 2 km away, is well out of
speci cation, our puri cation system produces water that exceeds the requirements of the
Daya Bay experiment.
Another challenge arises from the experiment environment, an open pool formed by blasting
granite in an underground cavern with personnel and equipment routinely present. Figure 2
shows the experimental con guration in the large hall. When lled with water, the pools
are covered with an opaque rubberized sheet, supported by a positive pressure layer of dry
N2gas, to keep out light and atmosphere. Given this environment, it is dicult to limit
contaminants that a ect water purity. For example, water resistivity at 25C can drop from
its theoretical maximum of 18.18 M
-cm to 17.5 M
-cm by dissolving 1.0 g/L of NaCl [8],
so cleanliness of preparation and constant ltration are necessities.
The remainder of this paper describes the design and performance of the water puri cation
system. We begin with our design goals, including a comparison to similar systems used in
other high energy physics experiments. We then outline our speci c system design for meet-
ing these goals, including a discussion of the monitoring hardware and software, and system
maintenance preparations. Next we show performance results, from both startup through
steady operation. Finally, we discuss the overall performance and suggest modi cations we
3
Figure 2: Cutaway schematic and photograph of the large (16 1610 m3) experimental hall. One of the
four antineutrino detectors is left out in each, and the photograph shows the pool partially lled with water.
Cherenkov light from cosmic ray muons is detected by the PMTs seen along the sides of the pool. There is
in fact a one-meter thick \outer pool", within the same excavated cavity, but optically separated from the
\inner pool" with vertical sheets of TyvekR
.
would pursue for a next generation system.
2. Water quality requirements and design goals
Ideally, one would design a water puri cation system based on allowed concentrations of
ion species which meet the optical absorption speci cations. Data is in fact available [9,
10, 11, 12, 13, 14] on absorption in water with di erent dissolved salts, and it indicates [12]
that the optical degradation is tolerable with signi cant amounts of what might be common
ions. However, we concluded that it was too dicult to anticipate what impurities might
be present, and to what concentrations, so our decision was to build a system which was
prepared to remove allions, leaving us with ultra pure water.
Consequently, existing systems from other high energy physics experiments [15, 16, 17, 18, 19]
provided criteria to guide the design of our system. These are summarized in Table 1, along
with a comparison to parameters for DYB.
Based on our site speci c constraints, and a comparison to the experiments listed in Table 1,
we settled on the following design criteria for water resident in the DYB pools: Particulate
Size1:0m; Total Dissolved Solids (TDS) 4 ppm; Resistivity 15 M
-cm; Dissolved
Oxygen0:1 mg/l; and Hydraulic Residence Time 7 days and 11 days, respectively, for
the small and large pools.
Naturally occurring40K and isotopes of uranium and thorium make the surrounding granite
radioactive. (No radioisotopes escape the containment vessels at the nuclear power plant.)
The water passively shields the ADs from the walls, but radon is a radioactive inert gas and
4
Table 1: Parameters of water puri cation systems used in previous high energy physics experiments, com-
pared to the DYB near and far pools.
Hydraulic
Volume Flow Rate Residence
Experiment (m3) (m3/hr) (gpm) (days)
BaBar/DIRC [15] 6 0.9 4 0.28
IMB [16, 20] 8000 18 80 18
SNO [17, 21] 1700 9 40 7.8
Super-K [18] 50,000 30 130 71
KamLAND [19] 3200 8 35 16.8
Milagro [22] 4000 43 190 3.9
DYB Far Pool 1996 9 40 8.3
DYB Near Pools 1232 5 22 6.7
can nd its way into the water through any number of pathways. Simple ltering will not
remove it, although activated carbon has been shown [23] to be e ective. However, the only
isotope of radon that produces a signi cant background for the neutrino experiment,222Rn,
has a half-life of 3.8 days, rather less than the water residence time in the pools. Therefore
our strategy for dealing with radon is to minimize its penetration into the water using a
non-gas permeable pool liner and allowing any residual radon to simply decay away.
Temperature control was a design requirement, as the water pool must be held between
21C and 25C, with a target of 22 :70:3C. This is critical for the operation of the ADs,
as they are lled with precise amounts of liquid scintillator mixtures and mineral oil, and
are equipped with over
ow tanks designed to handle the thermal expansion of the
uid
within this band. An operating temperature of 22 :7C was chosen as it matched the rock
temperature of the liquid scintillator production hall.
3. System design and maintenance projections
An initial engineering design was drafted at Brookhaven National Laboratory (BNL). This
design broke the system up into one Fill/MakeUp (FMU) stage that would feed each of
the three pools, and individual Polishing Loop (PL) stages in each hall. An RFQ based on
this design was sent to US and Chinese companies. We awarded the contract to Ultrapure
Environmental Engineering (Ultrapure), a company based in Shenzhen, China, and the initial
design was modi ed as the needs of the experiment became more clear. Figure 3 shows the
current layout of the FMU system feeding one of the three local PL systems.
The FMU is an 8 m3/hr centralized treatment system to produce ultrapure water for lling
and topping o of the pools. All water holding tanks in the FMU are polyethylene. The
layers of the media lter are 400 kg of anthracite 0.8 to 1.8 mm, followed by 1250 kg of
quartz sand 0.4 to 0.6 mm, then 195 kg of quartz sand 0.6 to 0.8 mm, and nally 195 kg of
5
Raw Water Tank
RO Product
Water TankRaw Water
Feed PumpMedia
FilterActive Carbon
FilterSoftenerRO Security
FilterHigh
Pressure
PumpNaOH
dosingpH
SensorTap Water
Conductivity
sensor
EDI Feed Pump 1Micron FilterRO
Conductivity
Sensor
0.1 Micron FilterPure Water
Booster Pump
P-31P-58EDIUV
TOC
DegasserReverse
Osmosis (RO)
Dissolved
Oxygen Probe
Resistivity
SensorTo Local Systems
From Fill/Make-Up SystemCity Water In
To Drain
To Drain
Vacuum TankHeater
Muon PoolCirculation
Pump 1 Micron
Filter0.1 Micron
FilterTemperature
Probe
Inlet Dissolved
Oxygen ProbeTemperature
Probe
Outlet Dissolved
Oxygen ProbeTOC UVMBDIDegasser Product
Resistivity
SensorReturn
Resistivity
SensorLevel
Sensors
C-starPool
Resistivity
Sensors
To Drain
To DrainChilled Water In
Chilled Water Out3-way Mixing
Valve~1000 m
Clean-PVCFill/Make-Up System
Local Polishing Loop
*#@$
&~Figure 3: Drawing of the central Fill/MakeUP (FMU) system feeding a local Polishing Loop (PL) system.
Water resistivity, our key performance parameter, is measured at the output of the FMU, and both at the
output and input of the PL. Note that the input to the PL is water from the pool during normal operation,
but is from the FMU during lling periods. Various monitoring points are marked with symbols to which
we refer in gures that follow.
quartz sand 0.8 to 1.6 mm. The activated carbon lter is 400 kg of activated carbon, followed
by 260 kg of quartz sand 0.8 to 1.6 mm, and then 150 kg of quartz sand 0.6 to 0.8 mm.
The Reverse Osmosis (RO) security lters are 5 m meltblown polypropylene. Total organic
carbon (TOC) removal is done via 185 nm UV lamp, and nal deionization via electro-
deionization units (EDI). Sterilization is accomplished with 254 nm UV lamps, followed by
1m and 0.1m lters made from folded membrane polypropylene. A degassi cation stage
is included just prior to the water entering the EDI.
The PL systems circulate at 5 m3/hr or (for the large pool) 9 m3/hr and are located in utility
rooms in each experimental hall for maintaining water quality via continuous circulation.
The 1 and 0.1 m lters are folded membrane polypropylene. The vacuum tank was rubber
lined ber glass, but it was eventually replaced with stainless steel when ruptures seemed
imminent. TOC removal is carried out via 185 nm UV, and mixed bed deionization (MBDI)
maintains resistivity. Sterilization is done with 254 nm UV, and a stainless steel stacked
6
plate cross
ow heat exchanger (HX) maintains temperature. A degassi cation stage sits
between the HX and the UV sterilizer.
The FMU system is connected to the local PL systems via several hundred meters of EslonR
Clean-PVC. Polyvinylidene
uoride (PVDF or KynarR
) and other types of piping are bet-
ter suited for use with high purity water, but the distances of piping needed ( 5000 m)
made the cost prohibitive. The entire system is controlled by a series of programable logic
control (PLC) units interfaced with a central computer, providing data collection as well as
operational control.
The FMU and PL systems use di erent deionization methods in the nal stage. Elec-
trodeionization (EDI) was chosen for the central processing system instead of mixed bed
deionization (MBDI) to reduce the need for chemical expendables, as EDI can bring the RO
product water to 18 M
-cm continuously without needing replacement or regeneration and
the original design anticipated that the majority of ion removal would happen at this step
of the treatment. Long term operations costs were considered in the choice of MBDI for the
local recirculation systems, instead of EDI. Unlike EDI, MBDI can maintain water quality
without electrical consumption beyond that of operating the pumps.
3.1. Fill/Make Up (FMU) System
The centralized FMU system consists of ltration, desalination, degassing, and sterilization
elements. Municipally sourced water enters the system via a 400PVC line and
ows into
a 5 m3polyethylene (PE) equilization tank. The source water then passes through one
of two Grundfos CRN15-3 booster pumps, with AISI 316 Stainless Steel (SS) throughout,
plumbed in parallel for redundancy via 200Chlorinated polyvinyl chloride (C-PVC) pipe
at 12 m3/hr. The water then passes through a 2 m3rubber-lined sand media lter tank
constructed of carbon steel with an empty bed contact time (EBCT) of 6.8 minutes, and
a 1.25 m3granulated active carbon lter tank of similar construction with an EBCT of
4.3 minutes, to be further re ned; removing potential particulates, biological, and chemical
contamination not removed during municipal treatment. The carbon is a coconut-based
granulated charcoal manufactured by Hainan Xingguang Active Carbon Co., Ltd.
The coarse ltered feed water passes through a 1.25 m3softening tank equiped, with Purolite
C100E sulfonate cation exchange resin based on a crosslinked polystyrene gel bead, removing
calcium and magnesium ions and preventing scaling on the RO membranes. Next a sodium
hydroxide dosing unit raises the pH of the water via injecting a 5-10% NaOH solution,
increasing the eciency of the RO unit. This is followed by a stainless steel (SS) security lter
housing tted with seven meltblown polypropylene (PP) 30005m lters to further decrease
the turbidity of the water and to protect the RO membranes from any large particulates still
in the feed water. The conditioned source water is then sent through a two stage RO unit by
a 17 m3/hr Grundfos CRCM15-12 high pressure pump with AISI 316 SS throughout. The
pretreated water is passed through two parallel Dunlop RO pressure vessels equipped with 4
Hydranautics CPA3-LD brine water RO membranes each. Product water from these parallel
7
units is transferred to a 5 m3RO holding tank made of PE, while the reject water is passed
through a nal stage of RO before being piped into the holding tank. This two stage RO
setup was desirable as it reduces concentrate water loses in the system to 23%, a concern
in system design. The product is a 9 m3/hr
ow rate of 1 to 0.1 M
-cm water. After the
RO unit all plumbing changes from C-PVC to EslonR
Clean-PVC, chosen for its low cost to
corrosion resistance ratio.
At this point the desalinated water passes through one of two parallel 9 m3/hr Grundfos
CRN10-6 booster pumps, with AISI 316 SS throughout, and begins the deionization, de-
gassing, and sterilization processes necessary to produce the high quality water needed for
DYB. The water passes through two parallel 300 Watt UV reactors to remove any organic
carbon still present in the water. The units are manufactured by RenownUV and are com-
prised of a 316 SS
ow cell with 185 nm UV lamps housed in quartz tubes. The units have
a contact time of 9 seconds rated for 5 m3/hr with 90% TOC removal eciency. This is
then followed by a 304SS lter housing equipped with seven 30 inch 1 m folded membrane
PP lters, installed to ensure the stability and security of the input water quality to the
downstream components. The water then enters four 6 X 28 Extra-
ow Liqui-celR
Mem-
brane Contactor degassing units, plumbed in a two by two fashion (two units in series, two
series in parallel).The membrane contactors contain several tightly grouped gas permeable
tubes around which the water is forced by internal baing. A single 1.5 hp Siemens pump
provides vacuum for each degassing unit, which draws dissolved gasses out of the product
water with a pressure di erential of -0.1 MPa gauge. The degassing units remove 95%
of the dissolved gasses from the water, but yet another stage is needed to bring the water
within the desired range.
This is immediately followed by three IONPURE IP-LXM30Z EDI modules plumbed in
parallel. The EDI runs continuously, producing water at 16 M
-cm consistently with 90
to 95 % permeate recovery. During a lling scenario, deionized water leaves the EDI unit and
ows through a 150 W ultraviolet sterilizing unit with a 254nm wavelength constructed with
316 SS
ow cell, designed for 9 m3/hr and with a 4 second contact time, to mitigate bacteria
growth in the processed water. The water then passes through one of two Grundfos booster
pumps rated for 8 m3/hr with AISI 316 SS throughout, before being forced through a 304SS
lter housing with ve 30000.1m bonded PP membrane lters. At this point it is referred
to as ultrapure water (UPW), and has a resistivity of approximately 18 M
-cm. (This is
consistent with ASTM Type 1 at this point, but as TOC, Na, Chlorides, Silica and Microbe
counts are not speci ed, we did not con rm they are present to the ASTM speci cation.) If
lling or topping is not taking place, deionized water is returned from the EDI to the RO
holding tank and is run on a continuous loop to maintain quality.
8
3.2. Localized Polishing Loop (PL) System
Each Experimental Hall (EH)1is equipped with a polishing system to further purify output
water and to maintain water quality during circulation. Water entering the polishing system
from the central system rst passes through a 304SS pre- lter housing optimally equipped
with ve 30001m PP membrane lters to protect downstream components from any large
particulates picked up in the pool. This is then followed by the rst stage of temperature
control in the system, a 50kW Berlin electric water heater, with 316SS internal plumbing.
The unit is set to heat the water to 25oC, slightly higher than the target temperature. It is
then cooled to the target temperature in a stacked SS plate Heat Exchanger (HX). Although
the source water is always too warm, this heating and cooling process provides a ner control
over the temperature than cooling alone.
This is immediately followed by a 185 nm UV TOC removal device of the same construction
and manufacturer as the ones used in the central system. This unit was installed to remove
internal sources of organic matter. Water is then passed through 2 MBDI units containing
100 L of resins plumbed in parallel. The MBDI units are lled with DowExTMmono-sphere
MR-575LCNG resins, a 1:1 cation:anion resin of sulfonic acid and quaternary ammonium
held in a styrene-divinylbenzene (DVB) matrix.
The MBDI is followed by a 254 nm UV sterilizer, identical to the unit in the ll make-up
hall, and implemented to mitigate bacterial growth during recirculation. This is followed by
a 304SS lter housing equipped with ve 2000(EH1/EH2) or 3000(EH3) 0.1m PP membrane
lters for removing sterilized bacteria, as well as removing any remaining particulates. Two
more Liqui-celR
degassing units in series follow this to further reduce dissolved gases in
the water, especially CO 2, O2, and Rn gases. The pools are optically and atmospherically
sealed with a cover constructed of US PTO Class 523 rubberized fabric with sulfur surface
processing. The cover provides shielding from the ambient hall light so that the PMTs
are able to detect the Cherenkov light produced by cosmic rays. Additionally, the cover
helps maintain water quality by preventing particulates from entering the pool and limiting
atmospheric gas dissolution. This is further enhanced by positively pressurizing the head
space with N 2gas from a 99.999% pure liquid nitrogen boil o .
The water is run through the HX to precisely hold the nal temperature to 22.7C before
entering the pool. The HX is fed water from the experiment's chilled water supply via a
three-way proportional valve adjusted using a feedback control loop. The water then
ows
into the pools and disperses through the inner and outer zones of the pool. The water is
returned to the system via a triple{tube siphon, returning water separately from the inner
pool, the outer pool, and the thin layer between the TyvekR
liner and the pool wall.
The siphon is drawn by one of two Grundfos CRNCM10-4 circulation pumps, with AISI
316 SS throughout rated for 10 m3/hr into a 0.2 m3vacuum tank with the inlet raised with
1The two smaller halls with neutrino detectors are called EH1 and EH2, and the large hall is EH3. For
historical reasons, the grotto that houses the FMU is called EH4.
9
respect to the outlet. These vacuum tanks were originally rubber lined ber glass, but were
eventually replaced with SS, for reasons described in Section 5.1. The polishing system
operates at 5 to 8 m3/hr and is capable of producing 18.2 M
-cm water with 10 ppb
dissolved oxygen.
The system was sized to provide 8 m3/hr during lling, and 5 m3/hr and 9 m3/hr, near and
far site systems respectively, during circulation. Approximately, 3 to 4 m3of water are lost
per day per experimental hall through the dissolved oxygen sensor
ow cells, resulting in
each pool being topped o once every 1 to 2 days.
3.3. The automated control system
The water systems are controlled by PLC units housed in each water utility room, and
connected to a central computer via Ethernet. The systems operate primarily via simple
feedback control loops connected to downstream sensors and monitors. In the central system,
a municipal water bu er tank level sensor controls the electronic municipal water entry ball
valve, creating a self-contained control loop. Similarly, the RO holding tank controls, at a
base level, the operation of the all the components between itself and the municipal water
equalization tank. Within this loop, a pH sensor controls the operation of the upstream
NaOH dosing unit, regulating the pH of water prior to reaching the RO unit. The RO
high pressure pump is controlled by pressure sensors upstream and downstream from its
operation, allowing necessary pressures for RO operation to be generated, while avoiding
damaging the membranes from excessive pressures.
In each of the local systems, three level sensors monitor the pool water level. Two of these
sensors provide level control, while the third is in reserve. In case one of the rst two fail,
this sensor will take its place as operation cannot cease to replace a sensor and access to the
pool requires data taking to stop. Temperature control within the systems is also managed
via feedback control from temperature relay units placed after the heater and the HX.
This system not only controls daily operations of the water system, but provides initial data
acquisition of many of the monitored parameters. This primary data acquisition system was
interfaced with the experiments detector control system (DCS) to allow remote monitoring
of the system's key parameters via internet. Parameters, such as pool outlet resistivity,
product resistivity, inlet and outlet DO, and temperature can all be plotted in real time and
are also archived.
3.4. System maintenance
Maintenance guidelines for the water systems were included in the operations manual pro-
duced by the system contractor Ultrapure Environmental Systems. Additionally, a main-
tenance contract has been signed with Ultrapure for the duration of the experiment for
the maintenance of the systems larger components and those requiring expert knowledge to
10
maintain. This contract covers the chemical cleaning of the RO, EDI, and degassing mem-
branes as well as the replacement of the MBDI resins in the local polishing loops. Ultrapure's
maintenance projections and contract covers the following:
Chemical cleaning of RO, EDI, and Degassing membranes every 6 months
Replacement of RO and EDI membranes and sand and charcoal lter media every 3
years
Changing of 5 m RO pretreatment lters every 15 to 30 days or when the pressure
drop reaches 0.10 MPa
Changing of 1 m EDI pretreatment lters every 3 to 6 months or when the pressure
drop reaches 0.07 MPa
Changing of 0.1, 1 m lters every 6 to 12 months or when the pressure drop reaches
0.07 MPa
Changing of the DowEX mono-sphere mix bed ion exchange resins every 6 to 12
months, or when the output drops below 14 M
-cm
Cleaning and calibration of all probes, sensors, and switchs every 3 months
Monthly inspection of all systems, even if there is no indication of problems, to reduce
the probability of unpredictable events.
UV system bulb cleaning, testing, and replacement as necessary
Emergency response within 24 hours
Providing a permanent or temporary solution within 14 days of incident, mitigating
system down time. It was determined that the experiment veto can operate without
water recirculation for this period of time with minimal ill e ects. This is discussed
further in Section 4.2
4. System Operation
Operating procedures evolved as we gained more experience. Each pool has been completely
lled twice and partially drained a number of times. Care was taken to prevent contamination
of the pool's surfaces from dust and other construction debris. A partial ll and drain was
executed prior to fully lling the pools in an e ort to remove any settled particulates from
the pool's lower surfaces. In spite of this, however, heavy particulate loads were experienced
at startup. (Details on dealing with this are discussed in Section 5.2.)
4.1. Testing
We veri ed the systems consistently produced water exceeding speci cations before they
were used to ll a pool. Additionally, each system was required to pass an operational
11
Jan 2012 Apr 2012 Jul 20120246810
pH
ConductivityConductivity or pHFigure 4: RO product conductivity (marked as @ in Fig. 3) compared to pH ($) of dosed inlet water of the
central FMU system, over several months. The vertical scale corresponds either to pH, or to conductivity in
S/cm.
readiness review performed by senior project management and engineering. As the FMU
was installed, the contractor tested the system after the installation of each new component
under manual operation. Once the FMU was installed, the automated system was tested
and tuned to eliminate any bugs in the software. We used the same process during the
installation of each local PL. The units were either operated with water passing to drain or
as a closed loop.
4.2. Process performance
The central FMU system has operated with only minor tuning since April of 2011, con-
sistently producing water with 17 M
-cm resistivity and 700 ppb O 2[aq] since the
system was rst fully operational. The high and consistent resistivity can be attributed to
the e ectiveness of the EDI, as well as the low conductivity produced by the RO. Figure 4
shows2how the RO removal eciency is directly linked to the upstream pre-treatment. An
unplanned increase in pH, in May 2012, directly leads to higher conductivity.
2Plots such as Figure 4 which show monitoring data over long periods of time, typically show a noisy
\grass" where data values changed drastically over very brief intervals. This is because the sensors are
located at junction points that sense di erent water quality when one of the three pools gets topped o with
water from the FMU, and ultra pure water from the FMU passes through the local systems on its way to the
pool, creating a distinct \spike" in the readings. Rather than arti cially remove these spikes, we emphasize
that one should focus instead on the trends in the bulk of the data.
12
Jan 2012 Apr 2012 Jul 201205101520Resistivity (M Ω−cm)EH1EH2
EH3Figure 5: Product resistivity (marked as ?in Fig. 3) of each of the local PL systems (EH1, EH2, and EH3)
from December 2011 to July 2012.
The PL systems consistently produce water with 14 M
-cm, except when the MBDI resins
were allowed to degrade to observe the e ect on pool water resistivity. Figure 5 illustrates
the dramatic drop in product resistivity that follows. This was accomplished by consciously
delaying the replacement of the MBDI resins until this signature asymptotic decline was
observed in an e ort understand mixing properties of the pool and the e ect of varied input
resistivity on veto performance and output resistivity.
A clear e ect on the average amount of detected light per cosmic ray event, can be seen when
circulation is stopped for extended periods of time. For two weeks in October 2012, the EH2
local polishing system was non-operational. Figure 6 shows the e ect during the one week
period after repairs were completed. The rise in pool water resistivity from 2 M
-cm
to3 M
-cm correlates with a 1 :5% increase in the average number of signal PMTs per
cosmic ray.
4.3. Resistivity Changes and Contamination from Pool Surfaces
The MBDI units raise the resistivity of the water entering the pools to 14 to 17 M
-cm
and the degassing units are capable of lowering dissolved oxygen (DO) concentrations to
10 ppb. However, water exiting the pools at the PL recirculation input has never been
larger than8:5 M
-cm, as shown in Figure 7. We presume that this reduction in resistivity
comes from contamination contributed by surfaces in contact with the pools, although the
mechanism for this is not yet understood. Possible sources include Fe ions leached from the
stainless steel due to a corrosion mechanism; dissolution of CaCO 3and SiO 3from concrete
13
15161718192021222324252627282922.53MΩ−cm
15161718192021222324252627282980818283
Date in October 2012Average Hit PMTsFigure 6: EH2 pool resistivity (measured at the recirculation input to the PL, marked as # in Fig. 3) and
the average number of triggered photomultipliers per cosmic ray, during a period in October 2012, after
repairs were made to the polishing loop. The correlation indicates an increasing attenuation length in the
water with rise in resistivity.
14
Jan 2012 Apr 2012 Jul 2012012345Resistivity (M Ω−cm)
EH1EH2EH3Figure 7: Resistivity of water at the PL input (marked as # in Fig. 3) for EH1, EH2, and EH3 for the same
time period as shown in Fig. 5. This is nominally the pool water resistivity, except for the spikes during
short periods of lling.
and construction dust; carbon dioxide dissolution due to poor sealing of the pool cover, and
bacterial metabolism supported by trace dissolved oxygen and dissolved organics leached
from system components. All of these potential sources of contamination have been examined
in an e ort to determine their e ect, and no single source can be identi ed as the primary
reason for somewhat degraded water quality.
Some clues to the nature of the contamination came accidentally. In early January 2012,
the water quality had not improved after months of circulation, including the persistence of
oating dust particles, although the cosmic ray veto performed adequately. It was discovered
that a bypass valve was mistakenly left partially open and only a sixth of the water was being
treated. This bypass was closed and as these particles were removed, both the water quality
and light collection improved. With time the resistivity of the water rose from 1 M
-cm
to2 M
-cm in EH1, and 4 M
-cm in EH2/3.
4.3.1. Metalic organic complex
In early March 2012, a buildup of yellow solids was discovered in the EH1 pre- lter housing.
Approximately 10 mL of this material was removed from the housing and sent to BNL for
analysis, along with sections of the lter from the housing. An X-ray Fluorescence (XRF)
analysis, shown in Figure 8, determined that the material was a metallic-organic complex
containing Fe, Zn, and Ag compounds qualitatively. Quantitative measurement of concen-
trations was not possible due to insucient sample size. However, an outside laboratory
15
0 2 4 6 8 1010−210−1100101
X−Ray Energy (keV)Intensity (Arbitrary units)MgAlAg
Ag
Ca
CaCr
CrFe
Fe
CoNiCu
CoZnZnFigure 8: XRF analysis of EH1 sludge sample taken March 2012.
determined that Pseudomonas aeruginosa was also present in the sample (contamination
during gathering of the sample is suspected due to the prominence of this microbe in most
environments).
It was decided that limiting sources of substrate and nutrients through continued treatment
of the water was preferred to other methods of treatment that would interfere with the
e ectiveness of the veto. For Example, IMB3had limited long term success in treating
Beggiatoa and Pseudomonas found in their system with peroxide. Subsequent samples taken
after this indicated decreased bacterial levels.
4.3.2. Carbon dioxide
We took 59 water samples, from di erent pools and di erent locations, to analyze for CO 2.
The samples were transported in biological oxygen demand (BOD) bottles, and analyzed
at Rensselaer Polytechnic Institute via gas chromatography with methanization and
ame
ionization detection (FID). Unfortunately, these samples failed to provide any conclusive
results. Measured Dissolved Inorganic Carbon (DIC) had signi cant variation within samples
from the same sample port, and no conclusions or trends could be deduced from the data
over all. This is most likely the result of a combination of low concentrations and imperfect
sampling technique. Indeed, a theoretical calculation [24] of the resistivity attributed solely
to aqueous CO 2at 22.5suggests that concentrations as low as 10-100 ppb (by weight)
3This was communicated to us as an internal report, from 1984, by Daniel Pope (Professor of Biology at
Rensselaer Polytechnic Institute, and President of Biotest, Inc.) prepared for the IMB collaboration.
16
Table 2: Data gathered from auxiliary resistivity sensors.
Date Hall Pool \out" RPI 1 RPI 2 BNL IHEP
M
-cm M
-cm M
-cm M
-cm M
-cm
01/30/13 EH1 1.40 5.08 5.09 5.11
01/31/13 EH1 1.45 5.25
03/12/13 EH1 1.35 1.3511.271
03/16/13 EH1 1.50 6.09 6.09 5.93
03/11/13 EH3 7.8 6.99 6.89 6.9 6.9
03/12/13 EH3 8.0 6.09 6.09 5.615.51
03/19/13 EH3 7.9 8.02 7.72
03/13/13 EH2 2.733.14
03/21/13 EH2 1.21 2.33
03/22/13 EH2 1.37 2.8
03/23/13 EH2 1.55 2.33
03/26/13 EH2 2.28 3.61
03/27/13 EH2 2.58 3.50
03/28/13 EH2 2.69 3.46
1Cross check of pool outlet sensor.
2System was stopped before a stable reading was made, and pool cover was removed.
3System was not recirculating, last stable value was on Feb 28.
would be enough to a ect resistivity at our observed levels. Furthermore, this method does
not distinguish between CO 2from atmospheric dissolution and microbial metabolism. Both
sources may have contributed to the concentrations necessary to lower the resistivity of the
pools.
Table 2 summarizes data gathered from additional sensors (detailed in section 5.2) that
provides insight into the resistivity drop that seems to occur within the pools. There is a
disparity between the resistivity of the water measured in the pool return line and that of the
water in the pool center, a disparity which is quite large in the case of EH1. We suspected
that irregular
ow and increased potential organics between the pool wall and the liner, a
semi-isolated circulation zone that might allow sucient microbial growth, could cause this
disparity. Subtle di erences in conditions at the pool walls could also account for the large
di erences between the di erent halls, as shown in Fig. 7.
To test the suspicion that the ultimate drop in resistivity came from microbe-laden pool
walls, we shut down physics data taking in EH2 temporarily, and installed a ball valve to
shut o water coming from the zone in-between the TyvekR
lining and the pool wall. The
resistivity from water returning from only the two pool sections was 3.5 M
-cm, signi cantly
higher than the 2 M
-cm resistivity (at the time of this test) from all three return lines. This
supports the idea that microbe growth on the pool wall is the source of the disparity.
17
Jul 2012 Aug 2012050100150200250300350Dissolved Oxygen ConcentrationIn
OutFigure 9: Dissolved oxygen content (in ppb O 2) measured at the inlet (marked as & in Fig. 3) and outlet
() points of the EH1 water system.
5. Multiple System Reproducibility and Process Modi cations
Though the local systems are identical in their components, order, and relative
ow/volume
ratios, a clear variation can be observed in the product resistivity (Figure 5) in each system,
as well as the dissolved oxygen content and resistivity (Figure 7) of the water returning from
the pools.There have been many suspected culprits investigated in an e ort to determine
not only why the water exiting the pools returns with such degraded quality, but what could
be the sources of these system to system variations. In the process of these investigations,
many bugs and minor problems have been discovered and corrected, though none could be
held entirely responsible for the di erences, or for the signi cant degradation of the input
water.
5.1. Equipment failure and replacement
In June 2012, it was discovered that the vacuum pump for the EH1 degasser was not operating
properly. Figure 9 shows that this was corrected in mid-July, with a noticeable e ect in the
outlet DO concentration. However, the pool was uncovered and partially drained before the
e ect on the return water could be observed. After the EH1 degassing unit was repaired, its
removal eciency was on par or better than that of the other halls.
During the run time of the experiment we have had a total of 4 level sensors fail in 3 separate
halls. The reason for these failures is still unknown. The original system design called for 2
sensors per hall, one primary conductivity sensor and a secondary hydrostatic level sensor.
18
In May of 2012, the hydrostatic level sensor in EH1 failed. Two hypothesis for the mode of
failure are power surges/poor grounding and moisture entering the atmospheric vent tube.
The decision was made to remove this sensor and install a replacement as well as additional
hydrostatic sensors in each hall during a planned experiment-wide shutdown.
Shortly after the experiment came back on line, the level sensors in EH3 began failing. The
conductivity level sensor began giving unreliable readings shortly after the water quality and
resistivity in the pool began to rise. By January 21, 2013 all three level sensors had failed
in this hall. After two months of estimation and manually lling the pool, ve new sensors
were installed. A 40 cm SS
oat sensor, a 50 cm conductivity level sensor, and 3 hydrostatic
level sensors (two in reserve) with atmospheric venting tubes.It was believed that the hydro
static level sensors failed because of humidity entering the atmospheric vent tube, which
was located under the pool cover. In order to prevent this from happening again, the new
hydrostatic sensors' vent tubes are located outside of the pool cover. It should be noted
that because of this, the di erence in pressure between inside the pool cover and outside can
translate to up to 12 cm di erence between measured and actual level.
On March 1, 2013, the EH2 vacuum tank split under the outward pressure of a leak
check/ lling. It was determined that the repeated pressurization of the vessel to check
for leaks was the cause of the fatigue and eventual failure of the tank. New stainless steel
tanks have been installed with additional pressure relief and gas purge valves to prevent
future failures.
5.2. Post installation upgrades
In early December 2011, an additional 316SS lter housing was installed in the local polishing
systems as a measure to protect downstream components from the previously mentioned high
particle densities, and was equipped with 1 m polypropylene membrane lters. While the
addition of these lters were previously proposed during the design process by engineers at
BNL (and a few companies that bid on the project) they were excluded from the contractors
design and the system until four months after the water system was initially started. During
this time the cost of changing several fouled resin beds and lters began to mount, and the
installation cost of an upstream larger pore sized security lter became more favorable.
In subsequent llings, we decided to start with larger pore size lters in an attempt to avoid
buildup of clogs, but still protect the MBDIs downstream. We used 2 m lters in the
security lter housing and 1 m lters in the downstream housing. The system operated like
this for several weeks until the lters ceased to experience signi cant pressure drops after
several days of operation. At this point, we resumed the use of 1 m and 0:1m lters.
After several months of working to diagnose the system behavior based on existing monitors,
we decided to install additional sensors to directly monitor the water quality. Three lines
were installed in each pool to take resistivity readings of the water directly with Rosemont
Endurance 400 conductivity sensors with 0.01/cm cell constants. These lines combine into
19
Jan Feb Mar Apr May Jun99.9599.9699.9799.9899.99100
Month in 2013Detection Efficiency (%)
EH1
EH2
EH3Figure 10: Measurements of the inner pool muon detection eciency over the rst half of 2013. Note that
EH3 is deeper in the mountain, so fewer muons penetrate, leading to larger statistical scatter.
a single line housing a Western Environmental Technologies Laboratory's C-Star transmis-
someter [25] before entering the local circulation system at the vacuum tank. These lines
operate using the same principle of the siphon return line used by the local system. Our
rst resistivity data is included in Table 2 under columns labeled \RPI 1" and \RPI 2".
Our initial transmissometer measurements do indeed show that that (blue light wavelength)
attenuation length is well above the 30 m speci cation.
6. Conclusions and Recommendations
We have demonstrated the ability, using only o -the-shelf technology, to build and operate
a water puri cation system in a hostile environment, that provides several thousand m3of
ultra pure water and reasonably maintains its purity through recirculation at several tons
per hour. The resulting water has a clarity high enough to work very well as a cosmic ray
veto system for a high energy physics neutrino experiment.
Ultimately, the measure of our success is the long term eciency of the water pool to actively
reject cosmic ray muons. Figure 10 demonstrates an eciency greater than 99.95% over many
months. Each AD can be used to identify a muon, and a corresponding signal is searched
for in the water Cherenkov data. Results are consistent in all three pools, are constant over
time, and this high veto eciency has been a key ingredient to a successful reactor neutrino
experiment.
20
The experiment, and the water system, are located in a relatively remote region in south-
eastern China. This necessitates a robust system that can operate with little intervention
and infrequent active maintenance calls. At this time, the system has operated essentially
continuously for more than two years, and is expected to continue operation for at least
another three years.
Some questions remain, mainly as to the nature of the small resistivity changes that occur
while the water is resident in the pool, between its production in the polishing loop and its
withdraw back into this loop. Several potential sources of this resistivity change are o ered,
but as they do not signi cantly a ect the operation of the experiment, these hypotheses have
not been rigorously tested at this time. More monitoring points would be recommended for
the next generation of such a system, and some way of continuously monitoring the water
absorption length directly could be implemented.
7. Acknowledgments
The Daya Bay experiment is supported in part by the United States Department of Energy,
oce of High Energy Physics.
References
References
[1] F.P. An et al. A side-by-side comparison of Daya Bay antineutrino detectors.
Nucl.Instrum.Meth. , A685:78{97, 2012.
[2] F.P. An et al. Observation of electron-antineutrino disappearance at Daya Bay.
Phys.Rev.Lett. , 108:171803, 2012.
[3] F.P. An et al. Improved Measurement of Electron Antineutrino Disappearance at Daya
Bay. Chin. Phys. , C37:011001, 2013.
[4] F.P. An et al. Spectral measurement of electron antineutrino oscillation amplitude and
frequency at Daya Bay. Phys.Rev.Lett. , 112:061801, 2014.
[5] F.P. An et al. The Muon System of the Daya Bay Reactor Antineutrino Experiment.
Nucl.Inst.Meth.A , To be published.
[6] RM Pope and ES Fry. Absorption spectrum (380-700 nm) of pure water .2. Integrating
cavity measurements. Applied Optics , 36(33):8710{8723, NOV 20 1997.
[7] FM Sogandares and ES Fry. Absorption spectrum (340-640 nm) of pure water .1.
Photothermal measurements. Applied Optics , 36(33):8699{8709, NOV 20 1997.
21
[8] K.R.. Morash et al. Measurement of the Resistivity of High-Purity Water at Elevated
Temperatures. Ultrapure Water , 11:18{26, 1994.
[9] E.O. Hulburt. The penetration of ultraviolet light into pure water and sea water. Journal
of the Optical Society of America and Review of Scienti c Instruments , 17(1):15{22,
JUL 1928.
[10] R.P. Buck, S. Singhadeja, and L.B. Rogers. Ultraviolet Absorption Spectra of some
Inorganic Ions in Aqueous Solutions. Analytical Chemistry , 26(7):1240{1242, 1954.
[11] L.J. Heidt and A.M. Johnson. Optical Study of the Hydrates of Molecular Oxygen in
Water. Journal of the American Chemical Society , 79(21):5587{5593, 1957.
[12] M. Ravisankar, A.T. Reghunath, K Sathianandan, and V.P.N. Nampoori. E ect of
dissolved nacl, mgcl 2, and na 2so4in seawater on the optical attenuation in the region
from 430 to 620 nm. Applied Optics , 27:3887{3894, 1988.
[13] W.S. Pegau, D. Gray, and J.R.V. Zaneveld. Absorption and attenuation of visible and
near-infrared light in water: dependence on temperature and salinity. Applied Optics ,
36(24):6035{6046, AUG 20 1997.
[14] James M. Sullivan, Michael S. Twardowski, J. Ronald V. Zaneveld, Casey M. Moore,
Andrew H. Barnard, Percy L. Donaghay, and Bruce Rhoades. Hyperspectral tempera-
ture and salt dependencies of absorption by water and heavy water in the 400-750 nm
spectral range. Applied Optics , 45:5294{5309, 2006.
[15] I. Adam et al. The DIRC particle identi cation system for the BaBar experiment. Nucl.
Instrum. Meth. , A538:281{357, 2005.
[16] R. Becker-Szendy et al. IMB-3: A Large water Cherenkov detector for nucleon decay
and neutrino interactions. Nucl. Instrum. Meth. , A324:363{382, 1993.
[17] J. Boger et al. The Sudbury Neutrino Observatory. Nucl. Instrum. Meth. , A449:172{
207, 2000.
[18] Y. Fukuda et al. The Super-Kamiokande detector. Nucl. Instrum. Meth. , A501:418{462,
2003.
[19] F. Suekane. KamLAND. Prog. Part. Nucl. Phys. , 57:106{126, 2006.
[20] George William Foster. An experimental limit on proton decay: p!e+0. PhD thesis,
Harvard University, 1983.
[21] Alan Dekok. Process degassi cation for SNO and radon removal. MSc Thesis, Carelton
University (1996) and private communication.
[22] R.W. Atkins et al. Milagrito, a TeV air-shower array. Nucl. Instrum. Meth. , A449:478{
499, 2000.
22
[23] A.I. Alabdula'aly and H.B. Maghrawy. Comparative study of di erent types of granular
activated carbon in removing medium level radon from water. Journal of Radioanalytical
and Nuclear Chemistry , 287(1):77{85, JAN 2011.
[24] T.S. Light, E.A. Kingman, and A.C. Bevilacqua. The Conductivity of Low Concentra-
tions of CO 2Dissolved in Ultrapure Water from 0 100C. In 209th American Chemical
Society National Meeting , April 2{6 1995.
[25] Sea-Bird Scienti c/WETLabs, PO Box 518, Philomath, OR 97370. C-Star Transmis-
someter .http://www.wetlabs.com/cstar .
23