Software Engineering Methods to Improve the Design of Software Reliability Systems: Roadmap

西 南 交 通 大 学 学 报

第 55 卷 第 3 期

2020 年 6 月

JOURNAL OF SOUTHWEST JIAOTONG UNIVERSITY

Vol. 55 No. 3

June 2020

ISSN: 0258-2724 DOI：10.35741/issn.0258-2724.55.3.27

Computer and Information Science

S

OFTWARE

E

NGINEERING

M

ETHODS TO

I

MPROVE THE

D

ESIGN OF

S

OFTWARE

R

ELIABILITY

S

YSTEMS

:

R

OADMAP

改善软件可靠性系统设计的软件工程方法：路线图

Idrees S. Kocher

Energy Engineering, Technical College of Engineering, Duhok Polytechnic University Kurdistan, Duhok, Iraq, idrees.hussein@dpu.edu.krd

Received: March 15, 2020 ▪ Review: May 16, 2020 ▪ Accepted: May 30, 2020

Attribution License (http://creativecommons.org/licenses/by/4.0)

Abstract

The reliability of software is founded on the development, testing, evaluation and maintenance of software systems. In recent years, researchers have been come to see software reliability as a major focus. This is due to the fact that reliability is central to all software quality concepts. System Reliability Engineering is the study of the processes and results of software systems in relation to the basic requirements of users. This paper provides an overview (roadmap) of current developments in software reliability metrics, modeling and operational profiles. It outlines several software engineering methods to achieve reasonable system reliability. Finally, failure metrics are considered based on feedback collected from users after releasing the software and case studies of detected failures. Consequently, numbers and types of failures will be recorded from the users feedback.

Keywords:Software Reliability Engineering, Reliability Objective, Operational Profile, Failure, Metric

摘要 软件的可靠性基于软件系统的开发，测试，评估和维护。 近年来，研究人员开始将软件可 靠性视为主要重点。这是由于可靠性对于所有软件质量概念至关重要。 系统可靠性工程是关于软 件系统的过程和结果的研究，与用户的基本要求有关。本文概述了软件可靠性指标，建模和操作 配置文件方面的最新发展（路线图）。它概述了实现合理的系统可靠性的几种软件工程方法。最 后，基于发布软件后从用户收集的反馈以及对检测到的故障的案例研究来考虑故障指标。因此， 将从用户反馈中记录故障的数量和类型。 关键词: 软件可靠性工程，可靠性目标，运营概况，故障，度量

第 55 卷 第 3 期

2020 年 6 月

JOURNAL OF SOUTHWEST JIAOTONG UNIVERSITY

Vol. 55 No. 3

June 2020

I. I

Software Reliability Engineering (SRE) plays a major part in the everyday operations of society, which is becoming more and more dependent on software systems. It is essential for systems to achieve the tasks within specified times without failures. SRE helps to achieve the required software quality for real systems. Systems have become more dependable and SRE research has continued to make significant progress in the past few years. This paper provides an overview of standards of Software Reliability (SR) based on models provided in [1]. The approach focuses on using principles of software engineering in software evolution and maintenance processes to enhance SR. The authors of [2] obtain new results that are helpful to understand the impact of the size of debugger teams on software debugging, failures of software, and other factors associated with reliability. SRE is considered a practical method because it is broadly applicable, cheap and has virtually no negative effects during implementation, as described in [3]. In [4], the aim is to introduce an overview of SR approaches or ideas, activities, modeling, metrics and improvement techniques. Ref [5] describes the various steps in failure data processing, aiming to observe and identify the evolution of software reliability. Ref [6] describes the history of SRE, present directions, problems and difficulties, and potential future trends in the field. Questions continue to arise because the complexity of software systems continues to grow with data size.

Software failure can cause serious consequences in everyday business operations. Based on literature, number of weaknesses in existing SRE methods was noted. At the start, the failed data will be combined through the next stages of testing (that is stages after unit and module are tested). This might be very late for performing any changes in the original design. Furthermore, is not enough to test the data that have failed and has been collected that are normally fails to work under actual and operational environment. The testing of software is believed to be a one of vital parts of improving the software reliability that can deal with these disadvantages which can become very important when delivering a quality product.

This paper will focus on improving and

developing the design of SR by using software engineering methods (techniques).

The rest of the paper contents are organized as follows. Section 2, debates the most available previous related works. In section 3, the term of software reliability is defined and explored. Section 4 offers a literature review of available metrics based software reliability. Section 5 outlines software reliability improvements based on software engineering techniques. Section 6 presents a process-based reliability simulation. Finally, Section 7 provides conclusions and suggested future works.

II. R

W

Software is considered a bottleneck in all systems development, with its costs and delay extensions placing many projects in danger. Computer software is the major source of reported errors for more than 80 percent of existing systems [7], while other errors including hardware make up less than 20 percent of total errors in existing systems [8]. Consequently these issues lead to revenue losses and expose human life to danger.

Reliability is the key factor of SRE. Software reliability (SR) is defined as the probability of fault-free software operation in a determined environment for a specific time period [9]. SR is one aspect of software quality in addition to several other properties that include the following: performance tasks, usability, capability, installability, maintainability, and documentation [10]. The producer should use quantitative measures to evaluate the quality of the product [11].

The SRE uses a quantitative approach that studies the operational behavior of software-based systems with respect to user reliability. The SRE approach includes the following components:

 The SR measurement, which includes software reliability, model-based estimation, and prediction;

 The aspects and metrics of the designed product, development steps, environmental operation of the software, and architectural system design; and

 Application of this knowledge to guiding and specifying system software architecture, testing, development, data collection, usage, and maintenance [12].

3

III. S

R

Product unreliability is generally based on a system’s failure. As software does not become qualified, its unreliability is mainly due to failures in the design that may be accompanied by errors in the software coding. Reliability is used to assess the performance of an entire system using known information about the system’s components and structure. Thus, reliability is the most vital characteristic in software quality. Software reliability is needed to handle the software functions are used for, to meet the needs of the customer, and to ensure the software product works on time and at an acceptable cost. The main goal of software reliability testing is to conduct many tests on documents, intelligence, tactics, code outline data, and trial data that support the measurement of reliability. Figure 1 shows the change in the rate of failure with the time of the operation. It also displays the upgrade effect in the SR performance in the period of useful life.

Figure 1. Failure rate change versus time

IV. M

B

S

R

The measurement of SR is still unsolved, due to a lack of understanding the nature of software. Thus, several metrics are utilized to measure the SR, which are presented in the following subsections.

A. Project Management Metric

It is essential to maintain a good management system to achieve the required project outcome with high quality. The cost of the project may increase, as designers no longer have enough processes. Enhanced and effective reliability can be provided by improving organized management, strategy management goals, risk managment processes and quality control.

B. Product Metric

The Lines of Code (LoC) and Kilos of LoC (KLoC) is a method used to measure the size of the software. Usually, unexecuted lines of comments and statements are excluded from the

underlying source code. The function point metric can measure whether the proposed software is functioning or not in the development process based on inputs, outputs, counting and interfaces. This may help to measure the essential operation and procedure of the software. The complexity-oriented metrics can find the program structure’s complexity, by simplifying the code and representing them in graphs, which can relate the complexity of the reliability of the underlying software. Therefore, the complexity representation of software is considered a vital product metric.

C. Fault and Failure Metrics

The Fault and failure metrics provide software with free failure execution. In this context, a vast number of errors can be found within the testing stage and before delivering the software product to the customers. Furthermore, the number of faults that occur as feedback from the customer after the software is distributed will be gathered, analyzed and outlined to complete the main goal of this stage. The basis of failure metrics is to gather feedback information from the users after releasing the software and on number of detected failures. Thus, the density of the failure will be found by using the collected failure data.

D. Process Metric

The process metric is used for approximation, monitoring and development of the quality and reliability of software.

E. Efficiency Metric

The needed time calculations and quantity of resources required for the software to achieve its specific function is an important factor in differentiating software. Therefore, an efficient software that is able to execute in a shorter period of time is considered to be high quality.

F. Integrity Metric

Integrity is an important issue when there is an existing hacking problem. Software that is accessed by illegal hackers can be controlled by improving the integrity techniques used to protect the software.

G. Flexibility Metric

This term is helpful when increasing the ability of software to be compatible with different types of hardware. In this context, software that is highly available and workable on different platforms is acceptable to most users.

H. Maintainability Metric

There is likely a robust relation between reliability and maintainability. Some specific features of maintenance activities should be emphasized. Maintenance policies and maintainability can be treated as separate aspects of system failure engineering.

Software often requires maintenance after it’s released. Preventive maintenance or corrective maintenance is carried out according to the system type, and both can have a very high cost. Software is generally not dependable without proper and timely maintenance. On the other hand, at times it can be more cost-effective to develop a less dependable system with proper maintenance. This can be more feasible than developing a more dependable system without maintenance.

V. S

E

T

B

S

R

I

To improve software reliability, it’s important to to apply appropriate software engineering methods. By using systematic and quantifiable techniques on the underlying development and maintenance of software, the outcome will be economical and reliable software that operates efficiently on real time machines [1]. Figure 2 illustrates the layered technology of SE that focuses on software quality and reliability.

Figure 2. Software engineering layered technologyapproach for high quality software development

A. Process Layered Technology

The approach in [1], [13], [14], [15], [16], [17], [18], [19] defines the process as a framework that is built to provide effective software engineering technology. This approach is considered the basis for software project management control, helping to establish the context for using technical methods, producing models and other works, establishing milestones, ensuring quality, and properly managing changes. For the best qualified software engineering, the

underlying process must be evaluated to meet the necessary principal process criteria.

In general, the integration between the software process and the software engineering methods used for improvement and evaluation are illustrated in Figure 3.

Figure 3. Integration between the software process and the applied software engineering methods in improvement and

evaluation tasks

B. Software Engineering Methods (Techniques)

Software improvement methods deliver an overall view of how to build software and cover the required technical aspects. Several tasks can arise with these methods, for instance, analysis requirement, modeling design, constructing the program, and trying the program. Figure 4 illustrates the basic stages for the development of the software as done in [1].

Figure 4. The process of software development

1) Analysis Requirement

When the development of the software was started, the same assertion was on coding and examining. However, researchers tried to show that analysis requirement was a very hard activity and prone to make errors. At this point, the failure rate of the software and the reliability could be increased. Thus, testing and coding are not sufficient for accomplishing acceptable reliability. Although analysis is considered to be very hard, it is important to reduce the rate of the failure, and reliability could be improved by

5

a) The identification requirement

b) Preparing and specifying the requirement of software (SRS) document whose goal is to describe the overall external behavior of the proposed system.

c) Validating the specific requirement of software (reviewing).

d) Prototype improvement.

e) Describing the structured analysis performance or the overall data flow of the software by forming Data Flow Diagrams (DFDs).

f) Estimate the task and the effort with its cost, and then assessing the resource.

g) Carrying out the risk assessment which may include risk management and control.

2) Modeling Design

Designing is the first phase in the process of moving from problem to solution. The main task of the design system’s model is to produce a model that may be used later to develop and use the real time system. Reliability may be improved through:

a) Dividing the system into small and identifiable parts, which may be developed and handled separately;

b) Removing some components to make maintenance easier;

c) Monitoring and considering the interdependency among the parts;

d) Reviewing the design to make sure that design puts the requirements and its quality;

e) Minimizing the conjugation between parts and maximizing the consistency within a part; and

f) Developing the design by repeating the process iteratively and regularly.

3) Program Construction

The coding process includes interpreting the design into the code during program construction. The reliability of the software may be improved by applying the following approaches:

a) Consider the code to be simple, readable, reliable documentation, and changeable;

b) Develop and adopt appropriate program styles of structured programming;

c) Consider developing an interface with other modules if implementing a modified reusable code; and

d) Perform inspections and code reviews to ensure quality and maintainability.

4) Testing

Software testing is done to validate the software product by distinguishing and eliminating defects since no software is 100 percent bug free. The main tasks of the testing phase include the following:

a) Confirm the quality of the created software; b) Show the correctness of the product functionality; and

c) Approximate the working reliability of the software.

Testing determines whether requirements have been fulfilled. Firstly, the system is separated into discrete units and each unit is isolated from the rest. Units are tested by the programmers and later testing may be carried out by independent testing groups with the goal of determining whether the unit is functioning as specified. After checking by units, the units are combined for integration and validation testing. Validation examining reduces the likelihood of defect incidences which may improve software reliability. Finally, testing may be performed for the entire system to improve software reliability. Figure 5 illustrates the effect of classifying and removing errors in the early stages of development on software reliability.

Figure 5. The effect of classifying and removing error on the software reliability

C. Software Engineering Tools

Computer Aided Software Engineering (CASE) tools provides guidelines for building a software product. CASE helps the software engineer automate manual activities and provides assistance for many tasks such as analysis, design, coding, and testing to conduct high quality with high reliable software.

VI. P

-B

R

S

Simulation indicates a method of imitating a specific set of circumstances to make quantified conclusions about an object or process. Reliability based on a software simulation process must mimic key features in order to validate and revise the code, and depends on accurate data for reliable modeling. However,

software projects cannot continuously gather data sets that are complete or reliable enough for efficient modeling research. In general, data needed for SR modeling is more problematic to collect than other software engineering data.

Because accurate data groups are rare, one purpose of a simulation is to control consistent data or software objects and use them to evaluate the assumptions for which reliability models are built. For instance, in real software, error correction and fault removal frequently disturb the assumptions necessary for reliability. However, simulation can provide a good understanding of such assumptions and might clarify why some models work more effectively than others.

Figure 6 explains briefly the steps of the simulation process. The first step would be to describe succinctly the problem to be solved. The definition of the problem essentially dictates the model to be adopted. At step one, it will become clear whether or not the form of the model permits analytical techniques to be used. Once the decision is made to simulate, the nature of the simulation technique employed will make it vitally important to decide which of many different parameters should be the major ones included. The program has to be written based on the number of iterations to be carried out and the order in which runs are to be made. The simulation should be in digital form on the computer. Once the model is established, its validity must be verified by a series of runs. When the initial results are obtained, several changes will probably have to be made to the model and the study’s plan. The early runs might make it obvious which parameters are indeed the most important, enabling programmers to reassess the model. After each run, the results must be verified and confirmed. Usually, it is helpful to repeat runs using different random

numbers each time. Figure 6. Steps of simulation process study

VII. C

F

W

The reliability of software can be considered one of the most important aspects of quality. Hardware can be eroded or become unworkable, but software cannot. Any unreliable behavior of a software system results from bugs in the development process. There is no single model that can guarantee the flawless behavior of a software system under all circumstances. Therefore, none of the existing models in the literature has been universally considered to work

7

perfectly for all different types of software systems.

One significant goal of developing any software system is to provide and deliver reliable software to the end customers. It is helpful to know what use customers plan to make of the system, since resources are not sufficient to test all aspects of it until reliability goals are established. This paper tried to deliver a simple overview or roadmap of SRE techniques to serve as a guideline for improving software system reliability. It also serves as a memorandum of the key points in each topic.

B. Future Works Statement

In future works, the SR-to-cost ratio will be addressed and developed, both within SR-related cost models and within SR models containing common-cause failures. This approach will lead to the reformulation of the Software Reliability Optimization Problem (SROP) as a combined integer-software issue.

A

This work is supported and granted by Duhok Polytechnic University at Duhok City, Kurdistan, Iraq.

R

[1] QUYOUM, A., DAR, M.–U.–D., and

QUADRI,

S.M.K.

(2010)

Improving

Software

Reliability

Using

Software

Engineering

Approach

-

A

Review.

International

Journal

of

Computer

Applications, 10 (5), pp. 41-47.

[2] LINET, C.-T., HUANG, C.-Y., and SUE,

C.-C. (2007) Measuring and Assessing

Software

Reliability

Growth

Through

Simulation-Based

Approaches.

In:

Proceedings of the 31st Annual International

Computer

Software

and

Applications

Conference, Beijing, July 2007. Piscataway,

New Jersey: Institute of Electrical and

Electronics Engineers, pp. 439-446.

[3] MUSA, J.D. (1999) Software Reliability

Engineering: More Reliable Software. Faster

Development and Testing. New York:

McGraw-Hill.

[4] PHAM, H. (2006) System Software

Reliability.

London:

Springer.

[5] KANOUN, K., KAÂNICHE, M., and

LAPRIE,

J.-C.

(1993)

Experience

in

Software Reliability: From Data Collection

to Quantitative Evaluation. In: Proceedings

of the IEEE International Symposium on

Software Reliability Engineering, Denver,

Colorado, November 1993. Piscataway, New

Jersey: Institute of Electrical and Electronics

Engineers.

[6] LYU, M.R. (2017) Software Reliability

Engineering: A Roadmap. In: Future of

Software

Engineering,

Minneapolis,

Minnesota, May 2007. Piscataway, New

Jersey: Institute of Electrical and Electronics

Engineers, pp. 153-170.

[7] GRAY, J. (1990) A Census of Tandem

System Availability between 1985 and 1990.

IEEE Transactions on Reliability, 39 (4), pp.

409-418.

[8] NATIONAL RELIABILITY COUNCIL

(1993) Switch Focus Team Report.

[9] INSTITUTE OF ELECTRICAL AND

ELECTRONICS ENGINEERS (1991) A

NSI/IEEE Standard Glossary of Software

Engineering Terminology. STD-729-1991.

[10]

GRADY,

R.B.

(1992)

Practical

Software Metrics for Project Management

and Process Improvement. Englewood Cliffs,

New Jersey: Prentice-Hall.

[11]

INTERNATIONAL

STANDARD

ORGANIZATION

(1991)

Quality

Management

and

Quality

Assurance

Standards – Part 3: Guidelines for the

Application of ISO 9001 to the Development,

Supply and Maintenance of Software. ISO

9000-3.

[12] LYU, M.R. (1996) Handbook of

Software Reliability Engineering. New York:

McGraw-Hill, IEEE Computer Society Press.

[13] PRIYA, P. and SHARMA, S.G. (2017)

Software

Reliability

Engineering:

An

Overview. International Journal of Science

and Research, 3 (6), pp. 553-556.

[14] CAI, K.Y. (1995) System failure

engineering and fuzzy methodology an

introductory overview. Fuzzy Sets and

Systems, 83, pp. 113-133.

[15]

VOUK,

M.A.

(2000)

Software

Reliability Engineering. In: Proceedings of

the Annual Reliability and Maintainability

Symposium,

Los

Angeles,

California,

January 2000.

[16] SRINIVASAN, N.K. (2000) Reliability

- How to Quantify and Improve? Improving

the Health of Products. Resonance, 5, pp.

55-63.

[17] LEVESON, N. (1995) Safeware: System

Safety

and

Computers.

Reading,

Massachusetts: Addison-Wesley.

[18] KANOUN, K. and SPAINHOWER, L.

(2014)

Dependability

Benchmark

for

Computer System. Wiley.

[19] CHI, D.-H. and KUO, W. (1990)

Optimal Design for Software Reliability and

Development Cost. IEEE Journal on

Selected Areas in Communications, 8 (2), pp.

276-282.

参考文

:

[1] QUYOUM，A.，DAR，M.-U.-D。和

QUADRI，S.M.K。（2010）使用软件工

程方法提高软件可靠性-评论。国际计算

机应用杂志，10（5），第 41-47 页。

[2] LINET ， C.-T. ， HUANG ， C.-Y 。 和

SUE，C.-C。（2007）通过基于仿真的方

法测量和评估软件可靠性的增长。在：第

31届年度国际计算机软件和应用程序会议

论文集，北京，2007年7月。新泽西州皮

斯卡塔维：电气与电子工程师协会，第

439-446 页。

[3] MUSA ， J.D. （ 1999 ） 软 件 可 靠 性 工

程：更可靠的软件。更快的开发和测试。

纽约：麦格劳-希尔。

[4] PHAM，H.（2006）系统软件可靠性。

伦敦：施普林格。

[5] KANOUN ， K. ， KAÂNICHE ， M. 和

LAPRIE，J.-C。（1993）在软件可靠性方

面的经验：从数据收集到定量评估。于：

1993年11月在科罗拉多州丹佛市举行的

IEEE国际软件可靠性工程国际研讨会论文

集。新泽西州皮斯卡塔维：电气与电子工

程师协会。

[6] LYU ， M.R. （ 2017 ） 软 件 可 靠 性 工

程：路线图。见：明尼苏达州明尼阿波利

斯市软件工程的未来，2007年5月。新泽

西州皮斯卡塔维：电气与电子工程师协会，

第 153-170 页。

[7] GREY，J。（1990）1985年至1990年

的串联系统可用性普查。电气工程师学会

可靠性交易，39（4），第 409-418 页。

[8] 国家可靠性委员会（1993）重点小组

报告。

[9] 电气电子工程师学会（1991）国家统

计局/ 电气工程师学会软件工程术语标准

词汇表。性病-729-1991。

[10] GRADY，R.B。（1992）用于项目管

理和过程改进的实用软件度量标准。新泽

西州恩格尔伍德悬崖：普伦蒂斯·霍尔。

[11] 国际标准组织（1991）质量管理和质

量保证标准–第3部分：在软件开发，供应

和维护中应用国际标准组织 9001的准则。

国际标准组织 9000-3。

[12] LYU，M.R。（1996）软件可靠性工

程手册。纽约：麦格劳-希尔，电气工程

师学会计算机协会出版社。

[13] PRIYA ， P. 和 SHARMA ， S.G.

（2017）软件可靠性工程：概述。国际科

学与研究杂志，3（6），第 553-556 页。

[14] CAI，K.Y。（1995）系统故障工程

和模糊方法入门概论。模糊集和系统，83，

第 113-133 页。

[15] VOUK，硕士（2000）软件可靠性工

程。于：2000年1月，加利福尼亚州洛杉

矶，年度可靠性和可维护性研讨会论文集。

9

[16] SRINIVASAN，N.K。（2000）可靠

性-如何量化和改进？改善产品的健康状

况。共振，5，第 55-63 页。

[17] LEVESON，N.（1995）安全软件：

系统安全和计算机。马萨诸塞州雷丁：艾

迪生·韦斯利。

[18] KANOUN，K. 和 SPAINHOWER，L.

（2014）计算机系统的可靠性基准。威利。

[19] CHI，D.-H。和 KUO，W.（1990）软

件可靠性和开发成本的最佳设计。电气工

程师学会通讯领域的期刊，8 (2)，第

276-282 页。