Trouble in River City again. The SVT05 crate has problems, which seem not to be fixed yet. Luminosity is low (18E30) so there's a 2 hour Level 2 Pulsar test in progress. Beam Division is going to dump the beam sometime this evening and try a new shot setup: I hope with more success than the feeble one last night.
The term SVT stands for Silicon Vertex Trigger. With tens of thousands of channels in the silicon system we need a rather complicated system for trying to figure out which hits line up into (curved!) tracks. It requires boards for gulping in raw hits and sorting them out, for trying to figure out if there are hits lining up in coarse roads, for trying to fit tracks for candidate collections, and for weeding out duplicates. We're in the process of upgrading this system to use narrower search roads, and everything needs to be tested: not just on the bench but also as part of the full system. And we want to make incremental tests: the Big Bang approach of wiring it all together and crossing your fingers is a wonderful recipe for chaos and confusion. You drown in error messages with no notion which system is the real culprit.
The weather map has massive blue claws reaching up around Aurora. Yes, weather does effect the detector. Pressure changes can change the drift speed of electrons in some of our drift chambers if the pressure is too extreme, and too much humidity encourages arcing and high voltage trips for some of the muon systems. In fact, we sometimes even have troubles with condensation. You'd think the massive air handling system would help, but apparently it isn't enough.
I live in an alphabet soup here. The Pulsar experts throw around obscure channel numbers and error flags in their conversations, but that's just the start. Et = Transverse energy, Pt = Transverse momentum (transverse means the vector component at right angles to the beam line), IMU = Intermediate MUon system, COT = Central Open Tracker, ed = Event Display, DAQ = Data AcQuisition, and on and on for pages. Somebody should update the CDF acronym table one of these days.
Eldest son would love the low voltage alarm. It starts with some kind of bird chirping. I'm not sure if it is real or synthetic, but he'd know. Not all of the alarms are real, of course: some are due to a failure to communicate with the crates in the collision hall.
The CMX (Central Muon eXtension) keeps tripping off its high voltage in miniskirt wedge number 30 in the NorthEast. The miniskirt got the name because this was the slightly below main floor level section; built later and with a different shape than the rest. When you look at drawings of the shape it is vaguely reminiscent of a miniskirt. A 6-foot long miniskirt.
One of the fairly annoying things about working here is that the Ace and/or experts will be muttering together facing away from the rest of us, make some decision or another to start/stop a run or reconfigure something, and neglect to notify the rest of us. The fans don't help, nor does the fact that most of the Ace/expert conversation is rather arcane.
The Pulsar testers are gone now, and we've about 2 hours before the beam store is dumped. No point in starting a checklist now, there's not been enough data accumulated in the histograms yet.
Checklist done. About a third of the items didn't have enough statistics to use, and something hung B0DAP56's window manager. Killing processes remotely didn't help, they were hung talking to the window manager. CTRL-ALT-BACKSPACE restarts the window w/o rebooting.
Beam dumped, but they're still tuning up for the new shot. We ran a calibration, but forgot it wasn't quiet time so it wasn't much use.
21:53, and still they're tuning the beam. I'd gone downstairs for a quick dinner (soup and beans and apples) and . .. still tuning. Belay that: "injecting final protons." ?? What are "final" protons? It didn't look like they had any bunches populated yet. (The particles race round the accelerator in 36 well-defined bunches.) 15 minutes later they've got 12 bunches loaded, but something seems to have gotten in the way. BTW, the BD has some cute sound effects: when they inject a proton bunch, they play a T-CH-SSSSSS sound over the monitors.
"pbar transfer. unstacking pbars" Followed by a transfer into the other beam line. Unfortunately they weren't able to fetch from the recycler, so this isn't going to be a record store.
Collider state: accelerating to flattop.
Collider state: flattop.
Collider state: squeezing.
"Event TimeOut: Partition 0, Content crate" The Ace has been on the phone to the experts for the past ten minutes. "The last hope is to start a new run!"
Tevatron scraping. Tevatron collimators moving to initial position.
Tevatron collimators: retracting proton collimators
Tevatron collimators: retracting ppar collimators
The shift only has another half an hour. Looks like we get beam, but the COT crates are giving us fits.
Collider state: high energy physics.
Luminosity is 70 E30. We're ramping the silicon--conditions must look clean. Getting an SVX alarm, though--maybe because it hasn't reached full voltage.
The run is going, no alarms; I collect more data in 10 minutes than the past three hours. And, of course, it is time to go.