During the discussions in preparation of the new versions of the International Electrotechnical Commission (IEC) standards IEC 61508-3 and IEC 61508-7, controversies regarding the proper roles of statistical validation or verification of safety-related software have emerged. These controversies regard changing demand profiles and continuous operation versus on-demand operation. This contribution derives a formula for calculating the failure probability per demand of software that has been tested under a demand profile that is different from the profile of its intended use. It also explains how failure rates can be expressed in terms of failure probabilities per demand, if the operational conditions are known. It further describes how software that is alternately operated continuously and on demand can be characterized in statistical terms and how the two operation modes can be recognized during a statistical evaluation. The notion of “mission” is suggested for sequences of demands or mixtures of demand-driven and continuous operation of software. In order to allow statistical calculations many requirements have to be met strictly. They are listed in the appendix. This article can hopefully facilitate licensing of software in many cases. Remarks are invited.
| Published in | Mathematics and Computer Science (Volume 3, Issue 4) |
| DOI | 10.11648/j.mcs.20180304.11 |
| Page(s) | 77-86 |
| Creative Commons |
This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited. |
| Copyright |
Copyright © The Author(s), 2018. Published by Science Publishing Group |
Software Safety, Statistical Testing, Operational Experience, One-Sided Confidence Interval, Changing Demand Profile, On-Demand or Continuously Working, Missions
| [1] | ISO/IEC 61508-1: Functional Safety of electrical/electronic/ programmable electronic safety-related systems, Part 1: General requirements (2010) Beuth Verlag Berlin or IEC Geneva. |
| [2] | IEC/TS 61508-3-1 Ed. 1.0: Functional safety of electrical/ electronic/programmable electronic safety-related systems - Part 3-1: Software requirements - Reuse of pre-existing software elements to implement all or part of a safety function, (2015), IEC Geneva or Beuth Verlag Berlin. |
| [3] | ISO/IEC 61508-7: Functional Safety of electrical/electronic/ programmable electronic safety-related systems, Part 7 Annex D (2010), IEC Geneva or Beuth Verlag Berlin. |
| [4] | Baldoni, Roberto; Giorgia Lodi, Luca Montanari, Guido Mariotta, and Marco Rizzuto: Online Black-Box Failure Prediction for Mission Critical Distributed Systems, 31. International Conference Safecomp 2012, LNCS 7612, pp 111-123. |
| [5] | Strigini, Lorenzo and Bev Littlewood, Guidelines for Statistical Testing (Report No. PASCON/WO6-CCN2/TN12). ESA/ESTEC project PASCON) London City University (1997) to be received via the authors. |
| [6] | Butler, Ricky W. and George B. Finelli: The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software; IEEE Transactions on Software Engineering, Vol 19, No1 (1993). |
| [7] | Littlewood, Bev and Lorenzo Strigini: Validation of Ultra-high Dependability for Software-based Systems, Communications of the ACM, 36(11), (1993). |
| [8] | Kuball, Silke; John May and Gordon Hughes: Structural Software Reliability Estimation; Safecomp 99, Lecture Notes in Computer Science, Vol. 1698, Springer pp 336-349. |
| [9] | Glaß, Michael; Heng Yu, Felix Reimann and Jürgen Teich: Cross-Level Compositional Reliability Analysis for Embedded Systems 31 International Conference Safecomp 2012, LNCS 7612, Springer pp 111-123. |
| [10] | Cotroneo, Domenico; Domenico Di Leo, Roerto Natella and Roerto Pietrantuono: A Case Study on State-Based Robustness Testing of an Operating System for the Avionic Domain, 30th International Conference, Safecomp 2011, LNCS 6894, Springer, pp 213-227. |
| [11] | Saifuddin, Ahmed: Methods in Survey Sampling Biostat 140. 640 – Stratified Sampling. pdf, (lecture notes) John Hopkins University, Bloomberg, school of public health (2009). |
| [12] | de Vries, Pieter G.: Sampling Theory for Forest Inventory – Stratified Sampling, ISBN 978-3-642-7 1581-5 (1986) Springer. |
| [13] | Gran, Björn Axel; Gustav Dahll, Siegfried Eisinger, Eivind. J. Lund, Jan Gerhard Norstrom, Peter Strocka and Britt J. Ystanes: Estimating Dependabilty of Programmable Systems Using BBNs, 19 International Conference Safecomp 2000, Springer LNCS 1943, pp 309-320. |
| [14] | Fares Innal: Contribution to modelling safety instrumented systems and assessing their performance – Critical analysis of IEC 61508; University of Bordeaux; Doctoral school of Physical and Engineering Sciences, presented 3rd July 2008; pp 49-53. |
| [15] | Bishop Peter G. et al.: STEM a project on Software Test and Evaluation Methods; Proceedings Safety and Reliability Symposium SARS 87, (1987). pp 100-117. |
| [16] | ISO/IEC 61508-4: Functional Safety of electrical/electronic/ programmable electronic safety-related systems, Part4: Definitions and Abbreviations (2010) Beuth Verlag Berlin or IEC Geneva. |
| [17] | Littlewood, Bev and David Wright: Some Conservative Stopping Rules for the Operational Testing of Safety-Critical Software, IEEE Transactions on Software Engineering, Vol. 23, NO 11, November 1997 pp 674-683. |
| [18] | Ehrenberger, Wolfgang: Operating Experience and Changing Demand Profile – Consideration of Paths, IFAC Congress 2014, Vol 19, pt1, ISBN 978-3-902823-62-5, pp 1619-1624. |
| [19] | Ehrenberger, Wolfgang: Nachweis der Funktionsfähigkeit von Software durch statistische Schlussweisen – Möglichkeiten, Bedingungen, Grenzen; Informatik Spektrum (2016) Springer Verlag, pp 384-392. |
| [20] | Eberhardinger, Benedikt; Hella Seebach, André Reichstaller, Alexander Knapp and Wolfgang Reif: Adaptive Tests for Adaptive Systems: The Need for New Concepts in Testing for Future Software Systems Gesellschaft für Informatik, Software Technik Trends Band 38, Heft 1 März 2018, pp. 61-64. |
| [21] | Hawkins, Richard; Alvaro Miyazawa, Ana Cavalcanti, Tim Kelly and John Rolands: Assurance Cases for Block-Configurable Software, LNCS 8666, Safecomp 2014, e-ISBN 978-3-319-10506-2, Springer, pp. 155-169. |
| [22] | Macher, Georg; Eric Armengaud, Eugen Brenner and Christian Kreiner: A Review of Threat Analysis and Risk Assessment Methods in the Automotive Context; LNCS 9922, Safecomp 2016, e-ISBN 978-3-319-45477-1, Springer, pp. 130-141 |
| [23] | Pyle, Ian: Developing Safety Systems, ISBN 0-13-204298-3, section 4. 5. 2, p 40. |
| [24] | From the discussions in the standardization group. |
| [25] | Tsong Yueh Chen and Yuan Tak Yu: On the Expected Number of Failures by Subdomain Testing and Random Testing; (1996), IEEE Transactions on Software Engineering, Vol. 22, NO 2. |
| [26] | Borges, Mateus; Antonio Filieri, Marcelo d’Amorim and Corina S. Pasareanu: Iterative Distribution-Aware Sampling for Probabilistic Symbolic Execution; ESEC/FSE’ (2015) ACM. 978-1-4503-3675-8/8/15/08 pp. 866-877. |
| [27] | Braband, Jens; Rüdiger vom Hövel and Hendrik Schäbe: Probability of Failure on Demand – The Why and the How; Safecomp 2009, LNCS 5775 pp. 46-54, Springer Verlag (2009). |
APA Style
Wolfgang Ehrenberger. (2018). Statistical Considerations on Software-Safety Estimation in Licensing. Mathematics and Computer Science, 3(4), 77-86. https://doi.org/10.11648/j.mcs.20180304.11
ACS Style
Wolfgang Ehrenberger. Statistical Considerations on Software-Safety Estimation in Licensing. Math. Comput. Sci. 2018, 3(4), 77-86. doi: 10.11648/j.mcs.20180304.11
AMA Style
Wolfgang Ehrenberger. Statistical Considerations on Software-Safety Estimation in Licensing. Math Comput Sci. 2018;3(4):77-86. doi: 10.11648/j.mcs.20180304.11
Share This Article
Twitter
Linked In
Facebook
Copy
Download@article{10.11648/j.mcs.20180304.11,
author = {Wolfgang Ehrenberger},
title = {Statistical Considerations on Software-Safety Estimation in Licensing},
journal = {Mathematics and Computer Science},
volume = {3},
number = {4},
pages = {77-86},
doi = {10.11648/j.mcs.20180304.11},
url = {https://doi.org/10.11648/j.mcs.20180304.11},
eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.mcs.20180304.11},
abstract = {During the discussions in preparation of the new versions of the International Electrotechnical Commission (IEC) standards IEC 61508-3 and IEC 61508-7, controversies regarding the proper roles of statistical validation or verification of safety-related software have emerged. These controversies regard changing demand profiles and continuous operation versus on-demand operation. This contribution derives a formula for calculating the failure probability per demand of software that has been tested under a demand profile that is different from the profile of its intended use. It also explains how failure rates can be expressed in terms of failure probabilities per demand, if the operational conditions are known. It further describes how software that is alternately operated continuously and on demand can be characterized in statistical terms and how the two operation modes can be recognized during a statistical evaluation. The notion of “mission” is suggested for sequences of demands or mixtures of demand-driven and continuous operation of software. In order to allow statistical calculations many requirements have to be met strictly. They are listed in the appendix. This article can hopefully facilitate licensing of software in many cases. Remarks are invited.},
year = {2018}
}
TY - JOUR T1 - Statistical Considerations on Software-Safety Estimation in Licensing AU - Wolfgang Ehrenberger Y1 - 2018/07/25 PY - 2018 N1 - https://doi.org/10.11648/j.mcs.20180304.11 DO - 10.11648/j.mcs.20180304.11 T2 - Mathematics and Computer Science JF - Mathematics and Computer Science JO - Mathematics and Computer Science SP - 77 EP - 86 PB - Science Publishing Group SN - 2575-6028 UR - https://doi.org/10.11648/j.mcs.20180304.11 AB - During the discussions in preparation of the new versions of the International Electrotechnical Commission (IEC) standards IEC 61508-3 and IEC 61508-7, controversies regarding the proper roles of statistical validation or verification of safety-related software have emerged. These controversies regard changing demand profiles and continuous operation versus on-demand operation. This contribution derives a formula for calculating the failure probability per demand of software that has been tested under a demand profile that is different from the profile of its intended use. It also explains how failure rates can be expressed in terms of failure probabilities per demand, if the operational conditions are known. It further describes how software that is alternately operated continuously and on demand can be characterized in statistical terms and how the two operation modes can be recognized during a statistical evaluation. The notion of “mission” is suggested for sequences of demands or mixtures of demand-driven and continuous operation of software. In order to allow statistical calculations many requirements have to be met strictly. They are listed in the appendix. This article can hopefully facilitate licensing of software in many cases. Remarks are invited. VL - 3 IS - 4 ER -
Information
Email: