Open Initiative for Next Generation of
Probabilistic Safety Assessment

I found the workshop very interesting and it confirmed the interest and the necessity for most of the people to have a standard for the PSA models. I found as well a positive reaction concerning the BDD technique even with the consideration of its limitations for some practical aspects.

In my opinion the developing of a standard and open source model representation will be essential for the future of PSA, as it is already a reality in other technical fields. It will bring many advantages for practitioners, software developers and researchers: clarity and transparency of the models developed, more direct and easy verification and comparison of the different results and techniques, and obtaining easily the models to construct a more wide benchmark to improve and validate the techniques and methodologies. If the methods and techniques can be tested through different models coming from different PSA software by reading its standard XML file, we could be able to improve or understand the different behaviors of the models and the techniques. This last issue of the benchmarking is a very important issue which is not always taken into account as so, but is an essential task needed to validate methods and approaches. I think that probably the most difficult part will be to understand the dependencies between the models and the PSA software and to integrate them in the standard. For this, is essential the participation of the software companies and the practitioners which indeed use this tools to develop the models.
It should be as well considered the incorporation of future techniques which may not be still mature enough for practical purpose but which are nowadays being investigated (dynamic methods, hybrid approaches, etc).

Concerning the BDD topic, I believe it offers improvements for the PSA assessment. A lot of effort has been done in the last years and it has produce positive results. Still, we are all aware as well of its limitations for its application to most of PSA models, so there is the necessity for more research on this area, both to continue to improve the technique and to consider some hybrid approaches as a next step to apply the methodology in PSA. Contribution between university research and the industry is essential in this context.

I think I will not be able to attend next meetings, as it is always a difficult task to obtain money for that kind of things from my university (!!), but I would like to be keep informed, as I found the project of the standardization very interesting and essential for the future application of PSA, and I was pleased to see more people working on the BDD (which is my PhD topic) and the positive reaction of some of the assistants to this.

I think that the workshop was definitely successful. In particular it was very interesting to see how different parties have tried to find common agreements with respect to future PSA software improvements.

I am working in the PSA field (especially in the operator modelling field using dynamic event tree) only for 1 year and I do not have as experience as the people who attended the workshop. By the way, I think that the idea of a standardized model composed by different tools would be helpful for new people who approach this field. Sometimes you really get lost in the jungle of the models!

From the point of view of new models and new calculation techniques I think that for the time being we cannot have algorithms that give accurate results simply due to 1) the long calculation time (we have to truncate somewhere) and 2) the problem of the uncertainties of the parameters. We need fast answers even if not highly accurate.

My personal opinion is that it would be nice to have a standardized dynamic model in which we combine the existing tools in a dynamic way, i.e. directly with the nuclear power plant model. But it should be probably a future step forward.

The idea to work out a standardized format for encoding models and to form a group aiming to develop the next generation PSA is excellent.
The transparency of models and data formats, benchmarking, validation of new modelling techniques, etc. are very important to improve accuracy of risk assessment.

Our Laboratory is interested to participate in this project, although we cannot contribute to all 6 areas (standard model, data model, visualisation, dynamic PSA, new calculation methods, pilot projects).
We as a research institute are prepared to take a part in the development of new evaluation techniques, e.g. agent-based modelling and to provide ideas for Pilot Projects.

We propose to set up a committee of very senior persons to give a steer to this important development project.
Make sure that the most critical parts of PSA, i.e. treatment of human factors, common cause failures and initiating events, uncertainties, are adequately addressed in the project.
Please put Wolfgang Kröger, professor and director (kroeger@mavt.ethz.ch) and Irene Eusgeld, senior scientist (eusgeld@mavt.ethz.ch), both of the Laboratory for Safety Analysis of ETH Zürich on the mailing list for further workshops/activities/events.

Advantages of the development of an open source PSA model representation standard

The development of an open soource PSA model representation standard provides many advantages for the PSA community and is therefore, also supported by NPP Goesgen as a user of PSA. The main advantages are:

• Improved quality assurance due to an increased transparency of models and the opportunity to verify results by applying alternative quantification and reporting techniques. A standard model representation will allow to use formalized mathematical validation techniques.
• Exchangeability of models will allow to improve the quality of models, and the model scope by using the best available techniques for the resolution of modelling problems. It opens the field for the development of new task specific applets to allow for example the incorporation of dynamic reliability techniques, the probabilistic treatment of deterministic phenomenological models ( e.g. in thermohydraulics, in structural mechanics, in human reliability analysis).
• An open source PSA model representation standard supports a faster implementation of new developments in science and technology (university research) into practical applications. The development of such a standard will contribute to an improved cooperation between university research and the industry.
• For utilities (customers) it opens the market for an improved competition between different consulting companies and PSA software vendors.

In GRS several different programs are used when working on a PSA level 1. These include among others RiskSprectrum PSA Professional, a MS Excel application to model CCF of large component groups, programs to do HRA, a program to compare fault trees, programs for fire PSA, a program for uncertainty analysis, programs for result visualisation, programs to create the input for PSA level 2, and others.
GRS is currently starting a new project to create a PSA management system to integrate all these programs within a common user interface. The aim of this project is to improve the quality of PSAs and especially to assure the traceability of the data flow in PSA projects. A further project objective is to create the ability to use the results of a PSA for a living PSA or precursor analysis.
A common standard file format to transfer data and results between these programs would greatly reduce the complexity of the project.

Based on your and Antoine’s proposal in Gösgen and the work done at GRS the following points should also be considered when drafting the standard (just brain storming, certainly not complete):
− For all data there should be the possibility to give reference to a data source, documentation, etc.
− For all calculation results there should be a reference to the used input data, the used software and the software version. Based on the given reference it should be possible to check if the results are still up to date (save a check sum of the input data, date of the file(s)?)
− The model has to be extensible: E.g. GRS is considering a coupling between the uncertainties of basic events depending on the data source they are based on. So all basic events have an additional parameter called coupling group. Might be another possibility: Store location information for basic events to be used in fire PSAs.
− The graphical ordering of fault and event trees should be saved in the file.
− The standard has to specify how the data is saved: E.g. in one big xml file, one xml file per layer (stochastic, …) in a directory, a subdirectory for each layer containing different files, …? Naming conventions for the files.
− All names of e.g. basic events, fault trees, event trees, etc. should be stored in UTF so that there are no problems with e.g. German umlauts.
− Store information about which programs have to be called to get certain (intermediate) results when calculating the PSA results. (Something like a “Makefile”).

I will not repeat the arguments that were given for a standard for PSA model representation, since I agree on them and they have already been put down in Jens Klügel reaction comment.

I will try to give additional comments on the proposal for the standard derived from my experience with PSA: review of existing PSAs and research.

From the review perspective, we are interested in how to improve the transparency and traceability of the PSA models. If a portable standardized PSA is produced, we could review it using the software which we feel more familiar with or that we feel that allows better traceability of the results. This may indeed improve the quality of the review.

On the other side, I realize there are some difficulties; in particular, it may be hard, or even not possible, to reproduce peculiar features of some PSA software in the representation standard, like the rules for house events (which may be important for external events analysis) or the maintenance & test models coded in the software. If a PSA model uses peculiar software features it may be difficult to transfer into the standard, unless some information is lost. And therefore, if we run it into a different software, we would risk looking into a model that is not the original one.

On the research side, one of my “dreams” is to manage to embed a PSA model (or part of it if the model is too heavy) into an optimization algorithm. This would allow rationalizing risk-informed decision for example on redundancies or inspection/maintenance times. Indeed, it is difficult to embed a heavy commercial software into another optimization program, as I would expect many interfacing problems (I never tried though). Using an open source code may help solving these interfacing problems.

Not much to add to what said above.
I am particularly interested in ways for PSA software to support decision making for safety improvement or any risk-informed decision (e.g. increase/relax inspection times? Put additional redundancies?). Visualization of data is an important aspect for this, since decisions are taken based on the information provided to the decision makers, and on how it is provided. In particular, the visualization of events’ importance results and of the effect of decisions on the risk profile may be of great help.
As I briefly discussed above in the position/discussion section, a parallel way to support decisions is to embed PSA models (or part of models) into optimization algorithms that would solve problems of choosing redundancy or test/maintenance strategies minimizing, for example, the expected risk.
Another comment: “support for decisions” is not mentioned as an area of interest in the announcement of this workshop (or at least not explicitly). I think it would be worth adding it, since taking decisions based on the results of the model is one of the most important goals for doing PSA.

While I was unable to attend the meeting, the concept is exciting. Currently it seems as of there are too many PSAs in too many models of varying capability. In some situations, comparing results is like comparing apples, oranges, kumquats, and pH of sea water.

While I’m sure that it is included in the above thoughts, I would like to see fully integrated PSA. Also, I would like to see provisions for standard outputs, tables, graphs, etc. Regulators, managers, and other non-daily PSA professionals do better with consistency.

As suggested by my comment above, an international standard PSA model would be ideal.

In my opinion, it would be a big advantage to have a standardized format for PSA models. However, I fear that the models of the different PSA tools are quite different in many details. Each PSA software has its unique and specific features which other tools have not. To bring this into a standardized format represents a huge challenge. However, we shall try this.

I was impressed by the advantages to solve a big PSA model using BDD techniques and without truncation. However, I have some doubts that we will ever be successful in having PSA quantification tools without truncation: We will always need fast answers with less accuracy and long-term answers of high accuracy. That means that the BDD quantification as the sole quantification tool will only be a very specific way to quantify a PSA model.
Actually in the event tree linking approach using RISKMAN, we are using BDD quantification of fault trees of single systems in conjunction with an event tree quantification that uses truncation. Therefore we have all options of different cutoffs.

This meeting was really interesting. All the presentation was new for me (never seen in congress). Because of the high diversity of participant the discussions was also really rich.

The proposition of a standardized format for encoding models seems to be really interesting.
A standardized format for output of PSA (MCS set and parameters) will allow PSA users to share their applications based on MCS set whatever their software (importance measures computations, uncertainty studies, Monte Carlo sampling…).
A standardized import format for PSA software will allow to compare PSA results and to developed common review tools. For example, a Risk Spectrum user will be able to review a PSA developed with CAFTA. This will facilitate external reviewing and cooperation between PSA analysts.

Furthermore, if each evolution of this international format during years is compatible with previous version, that will enable to archive PSA and it will be possible to open in 2020 a fifteen years old PSA.

Finally, if this format is developed and used in the nuclear field, it will probably spread to others industries (oil industry, chemic industry…) and to the academic world. It would become a common framework for discussion.

The meeting was definitively a success. I have been impressed by how different parties and interests sat together, trying to improve the quality of PSA codes and standards.

I think that having a standardized format for encoding models is essential for the future of PSA. It may not be as easy as one could imagine at first, especially when trying to develop a spanning format for different PSA approaches (e.g. linked fault trees / linked event trees), but much more difficult things have been standardized before. Since nuclear safety and PSA aren’t frivolous subjects, one can even wonder why a standardized format hasn’t been developed so far. Even if there are particularities between models, a PSA is nothing else than a relational structure of trees, distributions, conditions and rules.
Being able to benchmark approaches and codes is not only essential; it’s a question of seriousness.

I also think that time has come for a new generation of linked fault tree codes, maybe using BDD or some hybrid techniques. The first step is of course a standardized format.

A standard format may allow the following features :
- To benefit from other tools, non necessarily dedicated to PSA, from the wide developers community, that may facilitate some exploitation and maintenance operations on the PSA models. For instance, a standard format may allow :
o To have another way to validate the models and to check their consistency with their display through the fault tree or event tree interface.
o To easily manage models: compare versions of the same model, merge parts of different models, and manage thedevelopment quality (by, for example, querying parts of the model that don’t match QA criteria, for instance asumptions and tips that are not well documented).
o For PSA applications a major issue is about the development of different models, dedicated to specific applications. During their lifetime, these models change and a standard format may guarantee their convergence and prevent model errors and obsolescence.
o Visualization and navigation tools that may be very interesting to be familiar with the model, and even to review it

- From the PSA community
o The quantification with alternative tools may help to have strong confidence of the PSA results and be aware of the problems when there are any.
o Help to get hybrid approaches to handle dynamic phenomena using a combination with dynamic/simulation codes and facilitate batch programs for empiric computations.
o This may help to share some efforts and develop common tools.
o Encourage to exchange experiences.
o May help vendors and users to get benchmark instances to test versions

The development of the BDD tool presented by Mr. Nusbaumer was impressive. However, we still strongly doubt that a large scale PSA problem will be possible to solve without truncation and/or simplifications in a reasonable time frame. The future lies in a hybrid solution technique (RiskSpectrum C-BDD is an example of such a technique).

The presentation of the dynamic PSA was also very interesting. To include e.g. thermo hydraulics and human behaviour is interesting but it increases the complexity much. This really is a challenge!

We think that a standard format for exchanging information between different software is very good from a quality assurance aspect (verify results etc). This format should contain the necessary basic information.

We do not believe in a complete database format due to mainly two reasons:
• QA: we believe that there shall be no doubt about the format between the user interface software and the calculation engine. Changes in the user interface or calculation engine must walk hand in hand – otherwise it will be impossible for the users to actually provide evidence for a good QA to themselves or the authorities. THIS IS EXTREMELY IMPORTANT.
• A format that contain all information in the PSA database will slow the development of the tool down substantially (changes in the format must be agreed upon between all vendors). As an example, we are currently discussing changes in the definition of the success treatment within the event trees in RiskSpectrum PSAP. This is an example of a RiskSpectrum specific simplification and we want to have full freedom to change them (of course keeping our customers best interest in mind).

The aspects above will be much easier to handle if the interface is a basic format that contains the necessary FT/ET/basic event information. Then specific simplifications or techniques that are developed within a tool will not be important for other tools to follow.

I was very impressed by the work which was done (excellent presentation of Olivier Nusbaumer) and has to be done in risk analysis. It was a good idea to launch a platform for an unified format in risk analysis t couple all the different codes.

The workshop was really interesting for me and my company. We are discussing exact the same subjects and have some solutions which may be interesting for others, too. So we are interested in participation of further work of this group.
Concerning a general PSA model/format I think, that it would be possible to have a kernel and some rules how to expand the model. So it would be possible to interpret even individual increments. On the other hand, these increments will not be so individual, because we all do the same engineering work and use the same mathematics and statistics. So I think there will be a superset. But let us start with the kernel (FT and ET).

The improvements on BDD algorithm were very impressive to me. They lead me to the next research subjects on hybrid methods and dynamic variable reordering BDD method. I hope many parties such as utilities and software vendors participate in the activity as a member of working group. If all softwares take XML input standards, the users can easily test or convert their PSA models across PSA softwares.

The fault tree solvers could be categorized into
(1) BDD algorithm: fault tree > BDD structure (exact probability) > cutset (Dr. Rauzy's ARALIA, Mr. Nusbaumer's tool)
(2) ZBDD algorithm: fault tree > ZBDD structure > cutset (FTREX)
(3) Traditional Boolean method: fault tree > cutset (FORTE, RSAT, PSIMEX)
I hope all strengths and weaknesses could be reviewed and compared in the working group.

Name : Organization

Cristina Ibanez : Instituto de Investigacion Tecnologica.Universidad Comillas Madrid
Davide Mercurio : Paul Scherrer Institut
Irene Eusgeld : Laboratory for Safety Analysis, ETH
Göran Hultqvist : Forsmark Nuclear Power
Ulrich Hussels : RISA Sicherheitsanalysen GmbH
Jose Carretero : Empresarios Agrupados
Joachim Herb : Gesellschaft für Anlagen-und Reaktorsicherheit
Ola Backstrom : RELCON
Luca Podofillini : Paul Scherrer Institut
Martin Richner : AXPO/NOK
Nicolas Duflot : UTT
Olivier Zuchuat : BKW FMB Energie AG
Richard Attinger : Kernkraftwerk Gösgen-Däniken AG
Vinh Dang : Paul Scherrer Institut
Sigrid Wagner : Laboratory for Safety Analysis (LSA), ETH Zurich
Liao Yehong : Paul Scherrer Institut
Steve Epstein : ABS
Antoine Rauzy : ARBoost Technologies
Olivier Nusbaumer : KKL
Jens Klugel : Kernkraftwerk Gösgen-Däniken AG
Richard Quatrain : EdF
Mohamed Hibti : EdF
Woo Sik Jung : Korean Atomic Energy Research Institute
Amedee Barre : AREVA NP
Virgile Lalumia : AREVA NP
Shoba Rao : ABS
Don Wakefield : ABS