Integral Risk Assessment in Steel Ropes Diagnostics Using Computer Vision

Introduction. Timely assessment of technical condition of facilities directly responsible for safety of various technological operations is the most important task. This will reduce or eliminate emergencies with injury, loss of life and damage to material objects [1, 2]. Such critically important technical facilities from the point of view of risk assessment include: elevators (passenger, hospital and cargo); cable cars (passenger and cargo); mine hoisting installations; oil and gas drilling rigs; lifting cranes; hoists; ship lifting devices; skip hoists; rope mechanisms of offshore platforms; mechanisms on offshore pipelayers and others.

To carry out continuous monitoring of the parameters of steel ropes technical condition, a recommendation system of integrated risk assessment of the continuation of steel ropes work in mechanisms with the use and interpretation of computer vision system data has been developed.

The urgency of the problem is confirmed by the available statistics of accidents at facilities using cable traction, which occur despite the existing system of periodic inspections and flaw detection of steel ropes [3–5].

The work objective is to propose an integral method of risk assessment of operating machines with cable traction when defects are detected in a steel rope and combined with the use of computer vision during routine maintenance.

Tasks to be solved to achieve this goal in the course of this study:

1. Describe the existing regulations on visual and measuring control of steel ropes on machines with cable traction.
2. Propose a method of visual and measuring control using computer vision in a steel rope.
3. Show that computer vision is an artificial intelligence technology.
4. Define the intelligent decision support system (IDSS) in relation to the proposed method of risk assessment of operating machines with rope traction when defects are detected and combined with the use of computer vision in a steel rope during routine maintenance.
5. Apply one of the known methods of risk analysis for an integral assessment of the technical condition of steel ropes.
6. Propose an algorithm and a computational procedure for an integral risk assessment of operating steel rope with defects at a fixed length equal to six and thirty nominal rope diameters.

The starting point for the creation of the product was the emerging problems with elevators (and other cable cars) and the possibility of using video information analytics, which is based on deep learning algorithms for pattern recognition. This will significantly increase the objectivity of the information received about the parameters of technical condition of the steel rope, balancing devices, fastenings of the ends of the seals and ensure the level of safety of the operation of machines on cable traction, while eliminating the human factor [6, 7].

Materials and Methods. Steel ropes are subjected to daily, periodic and special inspections in all types of lifting structures. Operating instructions is the document used by the personnel during repair and maintenance work. In its absence, it is necessary to use Safety Rules or GOST standards.

Elevator electricians, crane operators, machinists, mechanicians, electricians and other categories of service personnel carry out visual inspections of steel ropes periodically or immediately before starting work. Inspections are carried out in the following order:

1. Familiarization with the entries in the journal (journal of the elevator daily inspection; watch — for the lifting crane; changeable journal — for suspended cable cars, etc.).
2. Inspection of the sections of rope wound on the drum, as well as those passed through the rope-carrying pulley, block, resting on shoes, fixed in couplings, couches and clamps.

The operator (driver) of the lifting structure has a log in which the results of inspections are recorded.

The frequency of flaw detection of steel ropes for specific lifting structures and their frequency are shown in Table 1.

Table 1

Frequency of inspections and flaw detection of steel ropes1

Rules¹ require periodic (in some cases, daily) visual and dimensional examination (VDE) of steel ropes with the detection of external defects, information about which is entered in the inspection log.

Defects in steel ropes arise as a result of:

• external influences with mechanical equipment or the external environment in the elevator shaft, including the formation of an electric arc discharge;
• inadequate quality of the supplied steel rope as a component product;
• improper quality of steel ropes installation;
• malfunctions of mechanical parts of the elevator, the indicator of which is the rope.

It can be stated that their defects in steel ropes are an "indicator" of technical condition of all mechanical equipment of the lifting structure, namely: wear of the pulley groove; rope slipping on the pulley, including as a result of excessive lubrication; misalignment of the winch attachment; backlash in the elements of the transmission mechanisms of the drive, etc. [8-12]. Possible malfunctions of the mechanical parts of the elevator, depending on the defective indicators, are presented in Table 2.

The VDE method is proposed², which makes it possible to carry out an integral assessment of the technical condition of a steel rope, implemented in the form of a software and hardware complex, which is a computational recommendation system for decision-making for service personnel when they perform maintenance work on machines with cable traction. This allows you to get the necessary safety level, while eliminating the subjectivity of the human factor, namely, physiological limitations associated with vision.

The proposed method provides for the creation, development and implementation of artificial intelligence, namely, a set of technological solutions that allow simulating human cognitive functions (vision and self-learning), including the search for solutions to identify the discovered defects in steel ropes with the results of human intellectual activity.

Table 2

Possible malfunctions of the elevator mechanical equipment depending on defects types

Artificial intelligence technologies, in accordance with "National Strategy ..."³ (subparagraph "a" of paragraph 5), include technologies based on the use of computer vision and intellectual decision support.

Artificial intelligence should be understood as a set of technological solutions that allows you to simulate human cognitive functions (including self-learning and finding solutions without a predetermined algorithm) and perform specific tasks with results comparable to or superior to the results of human intellectual activity.

The claimed method includes a set of technological solutions consisting of computer vision, information and communication infrastructure, software (including the implementation of machine learning methods), processes and services for data processing and solution search.

For an integral assessment of the steel rope technical condition, an intelligent decision support system (IDSS) using artificial intelligence (AI) methods was used (Fig. 1, 2).

Fig. 1. Functional diagram of the method of visual and dimensional examination of a steel rope (Authors’ figure)

An intelligent decision support system replaces a human consultant. It provides support to decision makers by collecting and analyzing factual data on detected defects, provides suggestions of possible options for action and their assessment. The purpose of artificial intelligence methods embedded in an intelligent decision support system is to allow a computer to perform these tasks, mimicking human capabilities as much as possible.

The proposed IDSS implementation is based on expert systems that encode knowledge and simulate cognitive behavior of human experts using predicate logic rules.

An expert system is a computer system that simulates the ability of a human expert to make decisions. Expert systems are designed to solve complex problems by reasoning with the help of a body of knowledge, presented mainly in the form of "if-then" rules, and not with the help of ordinary procedural code. The expert system is divided into two subsystems — the output mechanism and the inference mechanisms. The knowledge base represents facts and rules. The inference mechanisms apply rules to known facts to infer new facts. Inference mechanisms may also include explanation and debugging capabilities.

Fig. 2. Detection of defects by means of computer vision:

a — determination of external wires wear; b — wire breakage;

c — temperature effect; d — local decrease/increase in diameter

(Authors’ figure)

The created method makes it possible to automate processes when assessing the safety of using steel rope on cable traction machines. This makes it possible to provide users (staff) with a visual recommendation system for decision-making in the form of a web application that is not influenced by the human factor. This ensures increased security and reduced costs. This method automates and combines two control methods: visual and instrumental [14-16].

An expert system for risk assessment can be implemented based on well-known methods, such as[1]: Delphi method; failure mode and effects analysis (FMEA); failure mode effects and criticality analysis (FMECA); reliability-centered maintenance (RCM); risk indices; scenario analysis; structured method "What if?" (SWIFT); fuzzy logic methods, etc.

Regardless of the methodology used, risk assessment is carried out using a risk assessment matrix or a similar tool (for example, see GOST R 58 7714 or GOST R 55 234.3⁴, or GOST 12.0.230.5⁵).

The axes of the matrix can be both the probability and consequences of risk, as well as any other technical parameters of mechanical equipment that characterize their importance or criticality.

According to the results of such an assessment, risk is usually assigned one of three levels — high (red), medium (yellow) or low (green).

It is possible to increase the number of levels. The names of the colors are given as part of the most common scheme.

Standardization recommendations⁶ also provide for the use of a color scheme in risk assessment.

To assess risk levels and develop maintenance and culling strategies based on a recommendation decision-making system, we apply the RIMAP approach — procedures for monitoring technical condition and maintenance based on a risk-oriented approach (recommended by GOST 55 234.3⁵). The approach allows us to successfully combine all known methods of risk assessment and decision-making.

The application of the RIMAP approach solves the following tasks:

• identification of hazards;
• determination of the mechanisms of defects and failures;
• determination of the probability of failure;
• determination of the consequences of failure;
• risk level assessments.

The main danger for rope systems is the risk of rope breakage, as well as the destruction of the sealing points of the rope ends.

Table 3 shows the rejection rates of steel ropes and the mechanisms of damage development.

Next, you need to determine the probability of failure. This requires initial data on various defects listed in Tables 2 and 3.

Table 3

Rejection indicators of steel ropes and mechanisms of damage development

To obtain a reliable sample of data based on the available rope defect results, the Monte Carlo method was used, with the help of which a sample of 1000 rope sections was generated, multiples of its 6 nominal diameters, allowing considering all combinations and variations of defects. The estimation of the total probability of failure for the sample was carried out in accordance with the algorithm shown in Figure 3. The resulting sample was used to train a neural network in order to implement the risk indexing method of the RIMAP approach.

Since the output data is discrete, the neural network must solve the classification problem. A multilayer perceptron is selected as the type of neural network.

Fig. 3. Algorithm for estimating the total damage to the rope section (Authors’ figure)

During the training, 20 neural networks were obtained, from which the top five were automatically selected (Fig. 4).

Among the neural networks obtained, the MLP 11-9-3 network can be distinguished, which has the best performance on the test sample of 96 %, and on the training and control samples — 98 %.

Fig. 4. The best neural networks out of 20 trained ones (Authors’ figure)

These are the best indicators, therefore, for further work in the intelligent decision support system, we will use the specified neural network. This neural network has 11 neurons in the input layer (according to the number of defects at the input), 9 neurons in the hidden layer, as well as 3 neurons in the output layer (according to the number of possible risk assessments of further operation of the steel rope). A logistic function is used as the activation function of the neurons of the hidden layer, and a tangential function is used for the output layer.

Analysis of the sensitivity of the MLP 11-9-3 neural network to changes in influencing wear indicators showed that changes in the indicators J (elongation), I (torsion), E (local decrease in rope diameter) and G (waviness) have the greatest impact (Fig. 5).

Fig. 5. Sensitivity analysis of the MLP 11-9-3 neural network (Authors’ figure)

A certificate of registration of a computer program was obtained for the neural network program code7.

Results. An integrated risk assessment system has been created for the diagnosis of steel ropes using computer vision, which allows detecting defects in steel ropes in a timely manner, assessing the existing risk of further operation and giving recommendations to specialists of operating organizations in real time. This will dramatically reduce the risk of accidents, injury and death of people at facilities using steel ropes.

This system is the core of the product — a software and hardware complex (PAC) (Fig. 6) automating the processes of assessing the safety of using steel ropes, defects in balancing devices and places of sealing their ends on machines using cable traction, which is a visual recommendation system for decision-making for specialists. At the same time, the human factor is eliminated, which ensures increased safety and reduced costs.

The main functional purpose of PAC is the automatic detection of defects in steel ropes, balancing devices and fasteners of the ends of rope seals by special optical means (photo and video fixation), followed by processing of the received digital information by photo and video analytics based on machine learning, including an integral assessment of suitability for further operation by risk analysis methods, which is translated into the risk assessment color scheme on users' mobile devices.

Fig. 6. PAC functional scheme (Authors’ figure)

Discussion and Conclusion. Integral risk assessment in the diagnosis of steel ropes using computer vision, taking into account its advantages over the traditional visual and instrumental control system will be in demand by the industry of cable traction machines. This market is the target for the use of the product created by the authors. For example, the number of passenger elevators in operation on the territory of the Russian Federation is 450 thousand units. More than 50 thousand organizations are engaged in elevator maintenance. Each elevator requires a hardware part of the complex, i.e. the volume of the consumer market is 450 thousand pcs. The software part is necessary for each service organization, i.e. 50 thousand licenses. The mobile application is necessary for the specialists of each service organization, which must have at least three specialists, i.e. 150 thousand licenses. Taking into account the technological and economic advantages (the price is about 55 thousand rubles), the developed system will take a strong position in the industry of cable cars.

¹Ob organizatsii bezopasnogo ispol'zovaniya i soderzhaniya liftov, pod"emnykh platform dlya invalidov, passazhirskikh konveierov (dvizhushchikhsya peshekhodnykh dorozhek), eskalatorov, za isklyucheniem eskalatorov v metropolitenakh. Government of the Russian Federation. Electronic fund of legal and regulatory documents. Available from: https://docs.cntd.ru/document/436745439?section=text (accessed 23.01.2023). (In Russ.).

²Korotkiy A. A. et al. Sposob vizual'no-izmeritel'nogo kontrolya stal'nogo kanata. Patent 2775348, Russian Federation, D07B 1/00; B66B 7/1215; G05B 99/00. No. 2021107842, 2022. 16 p. (In Russ.).

³O razvitii iskusstvennogo intellekta v Rossiiskoi Federatsii (vmeste s «Natsional'noi strategiei razvitiya iskusstvennogo intellekta na period do 2030 goda»). President of the Russian Federation. Electronic fund of legal and regulatory documents. Available from: https://docs.cntd.ru/document/563441794/titles/64U0IK (accessed 23.01.2023). (In Russ.).

⁴GOST R 58771-2019. Risk management. Risk assessment technologies. NPO RusRisk, Technical Committee for Standardization "Risk Management", Federal Agency for Technical Regulation and Metrology. Available from: https://docs.cntd.ru/document/1200170253 (accessed 23.01.2023). (In Russ.).

⁵GOST R 55234.3-2013. Practical aspects of management of risk. Risk-Based Inspection and Maintenance Procedures. ANCO "NITS KD", Technical Committee for Standardization TC 010 "Risk Management", Federal Agency for Technical Regulation and Metrology. Electronic fund of legal and regulatory documents. Available from: https://docs.cntd.ru/document/1200108150 (accessed 23.01.2023). (In Russ.).

⁶GOST 12.0.230.5-2018. Occupational safety standards system. Health management systems. Risk assessment methods to ensure the safety of work. Federal Agency for Technical Regulation and Metrology, Interstate Council for Standardization, Metrology and Certification. Electronic fund of legal and regulatory documents. Available from: https://docs.cntd.ru/document/120016046  (accessed 23.01.2023). (In Russ.).

⁷R 50.1.090-2014. Risk management. Key risk indicators. NP RusRisk, Technical Committee for Standardization TC 010 "Risk Management", Federal Agency for Technical Regulation and Metrology. Electronic fund of legal and regulatory documents. Available from: https://docs.cntd.ru/document/1200120834 (accessed 23.01.2023). (In Russ.).

⁸Korotkiy A. A. et al. Integral'naya otsenka tekhnicheskogo sostoyaniya stal'nogo kanata : progr. dlya EVM. Cert. No. 2022683712. Inzhenerno-konsul'tatsionnyi tsentr «Mysl'». No. 2022683761, 2022. (In Russ.).