It would therefore be beneficial to have a framework to automatically test and validate the software and data of an experiment against changes and upgrades to the environment, as well as changes to the experimental software. As such a framework would examine many facets common to several current HEP experiments interested in a more complete data preservation model, the development of a generic validation suite is favourable.
My credentials: I spent several years working with code management and validation on CDF, and the last project I worked on had, at my insistence, a focus on making the release as portable as possible so that colleagues could continue to analyze data for years to come, without worrying about changes in operating system.
It took months to validate an ordinary release--there were always little gotchas, and code that used to work, and things that changed when the underlying system libraries changed. The last project is still in progress :-( after a year.
It would be wonderful to have an automatic testing framework, but it would be hugely complex and there's no way it could be generic. And software upgrades (bug fixes) are anathema to people want a stable framework for analysis and comparison. (If both the simulation you did two years ago and this year's data processing share the same bugs you can still compare the results.) The releases also have to be vetted to make sure the physics results make sense. The last time I heard CMS and Atlas still hadn't validated their code on the SL6 operating system either.
There are two nightmares: when something triggers a bug in the compiler and when "the physics results diverge." The first eats a lot of expensive talent's time and wastes a lot of everybody else's as you converge on an agreement that the compiler is at fault. The second involves getting some very busy scientists to step through a lot of tedious debugging and then argue about whether the result is an improvement and if it is, is it a big enough improvement to disturb the status quo.
Yes, most of the analysis software was written by scientists. They have the domain knowledge, but it doesn't always translate into clean code. (And C++ evolved, and some people learned to do it one way and others another. And some people tried to get cute. And so on.)
To be fair, the paper goes on to say that HERA has a test version of a test suite, and if it is built in from the get-go it is probably easier to do.
2 comments:
And to think all I have to worry about is favorite old movies trapped on VHS tapes for which I no longer have a player.
Nobody recorded family Christmases? (Us either, except for a few cassettes we sent to absent relatives)
Post a Comment