System for Identifying the Condition of Hoisting Crane Runways

Introduction. The inspection of overhead crane runways in production facilities and warehouses is a complex and risky task for experts and professionals in the field of lifting equipment. Currently, the inspection of overhead crane runways is conducted in accordance with Rostechnadzor Order No. 461 dated November 26, 2020 “On the Approval of Federal Standards and Regulations in the Field of Industrial Safety “Safety Rules for Hazardous Production Facilities with Lifting Facilities”². The inspection involves visual and measuring control along the entire length of the crane runway, as well as geometric parameter control during the planned compilation survey using special devices such as total stations and theodolites. The danger during this inspection arises from the fact that crane runways are located at a considerable height, usually ranging from 3–5 meters, but in some cases exceeding 20 meters. Overhead cranes come in two types, depending on how they are supported on the crane tracks. There are supported cranes and suspended cranes. In the supported type, the runways are attached to the crane beam, which is most often an I-beam, by welding or bolting, and these joints are prone to wear. When heavy-lift cranes are installed, connecting passages may be arranged to allow for inspection of the supporting crane runways. However, if the lifting capacity is less than 10 tons, connecting passages may not be provided. With suspended cranes (where the crane runways are in the form of I-beams), connecting passages are also typically not provided. In such cases, inspection is conducted from repair sites and using scaffold towers, which greatly increases the difficulty and duration of the inspection, as well as the risk of falling from height. Therefore, it is necessary to develop examination methods that do not require specialists and experts to be lifted to a height, while ensuring the accuracy of detecting and evaluating defects. Additionally, there is a challenge in developing faster and more efficient control methods than direct visual inspection.

The scientific literature has already discussed the potential of artificial intelligence for industrial safety. One well-known platform is a digital video monitoring system [1], which provides round-the-clock monitoring of crane operations, loading and unloading activities. It also monitors the use of special clothing and personal protective equipment by workers, the presence of workers in a dangerous area when moving goods, areas of rail tracks (absence or presence of unauthorized persons, equipment), the condition of load-handling devices (LHDs), as well as the positioning of transported cargo, and compliance with warehousing regulations. The system uses video analytics from IP cameras, processed by artificial intelligence, to provide this information to specialists and managers. The advantage of this approach is that it allows for comprehensive control over crane operations and personnel actions and equipment. This method is based on video data from permanently installed IP cameras. Each camera covers a fairly large area, although the exact size is not specified [1]. This allows the method to perform its intended functions. However, it is not suitable for overhead crane runways. To identify local defects, the video must cover the entire length of the runways and have high resolution. Shooting must be done in stereo format, with data on the depth of the structure and defects.

A method for LHD assessment based on computer vision algorithms trained using the Roboflow platform has been developed and applied [2]. The method allows for assessing the condition of LHDs located on a storage stand by processing photo and video materials captured by a webcam. However, the limitation of this method is that it can only be applied to LHDs as the webcam is permanently installed on a special stand for storing them, and it cannot receive data from a significant distance. Additionally, the neural network used for training was focused on defects specific to LHDs and therefore cannot be used for overhead crane runways.

In foreign studies [3], computer vision neural networks based on the pre-trained YOLOv8 model [4] and complex data analysis of the large GPT-3.5 Turbo language model [5] were used to ensure the safe operation of tower cranes. The authors developed a specialized safety knowledge base, which was compared with GPT-3.5 to discuss safety in context. The model proposed in [6] provides practical recommendations for improving safety during the operation of tower cranes in high-risk construction. The authors of [7] claim that computer vision can provide real-time spatial awareness, dynamically determining worker positions, equipment movement, and proximity to hazardous areas. A positive aspect of this study is its emphasis on staff safety and the recommendation of AI tools to further improve it. However, computer vision is unable to cover the entire working area of a crane, leaving out possible local structural defects. This makes it impossible to use computer vision methods to identify defects in the tracks of overhead cranes.

It is known that unmanned aerial vehicles (UAVs) are used for visual inspection of hard-to-reach areas of tower crane structures [8]. This allows for the identification of defects and damages, as well as the prevention of accidents. While this survey method has many advantages, it should be noted that the UAVs used for this purpose are only intended for outdoor use, as their positioning is based on GPS navigation protocols. Overhead crane runways are typically located inside buildings (workshops), so this method cannot be used to examine them.

The authors of [9] use computer vision methods to determine the degree of wear of parts, corrosion, and residual life of machines.

There are studies on the integral assessment of the risk of operating steel ropes using computer vision technologies [10]. Based on the analysis of possible defects in steel ropes [11], a neural network computer vision model [12] was trained and an intelligent decision support system (IDSS [13]) with artificial intelligence methods was developed. This development has several advantages, and its principles can be applied to develop methods for inspecting overhead crane runways. During this development, a database of rope defects was created by artificially applying them to 1,500 lengths of rope, followed by stretching them under the cameras on a horizontal milling machine. The result of this work is a stationary device that includes elevator ropes (or one cable car rope) with three webcams and a light source inside the device. The ropes move inside the device under the continuous all-round surveillance of the cameras. This principle of operation is not suitable for the inspection of overhead crane runways.

There are several UAVs that are designed specifically for use indoors and in enclosed spaces [14]. These UAVs can be equipped with cameras of various types and spectral ranges of operation, as well as lidars to create a point cloud of the surveyed objects [15]. A positive feature of UAV data is its independence from GPS navigation. At the moment, UAV-based methods are used to survey difficult-to-access enclosed spaces, such as blocked sections of mines or rubble of destroyed buildings, to adjust rescue operations. While these studies do not involve the use of artificial intelligence for the automated detection of defects in overhead crane runways, some ideas from these studies could be applied to the development of a method for examining overhead crane runways.

An analysis of the published results of such studies has revealed a gap in scientific knowledge regarding the lack of an effective survey method for crane runways located at a height inside industrial premises. This method should identify defects in crane tracks with sufficiently high accuracy and inspection speed, while ensuring the safety of personnel. This method could be developed by integrating the scientific knowledge presented in previous research, adding unique and essential components.

This research aims to develop a method for remote inspection of overhead crane tracks located at a height in industrial premises, which will minimize the impact of hazardous and harmful production factors on technical personnel and ensure the accuracy, speed and reliability of the results.

To achieve this goal, it is necessary to solve the following tasks:

1. Create a three-dimensional point cloud for the rails, crane beams, supports, with reference to the locations of crane beam supports along the axes of both rails (hereinafter referred to as the A-axis rail and the B-axis rail). The model should provide a complete picture of the relative positions of rail elements along these axes.
2. Add a three-dimensional cloud of high-density points for A-axis and B-axis rails to identify deviations of crane runway elements from their design positions in plan and profile (planned high-altitude survey), as well as detect defects in crane tracks.
3. Develop a three-dimensional model of the above-ground crane runways with automatic assessment (measurement) of deviations in the geometry of crane runways, wear and local defects, and link them to column lines and sections.

Materials and Methods. Data on defects on overhead crane runways was used as initial information. This data was obtained during the inspection of overhead cranes as part of expert control of lifting cranes at machine-building enterprises in 2024. A survey was conducted during which 352 overhead crane runways with a total length of 14 kilometers were examined. The methodological part of the study was based on GOST R 56944–2016³. Computer vision neural networks were trained using open libraries for the Python language. An upgraded YOLOv8 pre-trained neural network was used to detect defects. Its architecture was a three-part system consisting of a feature extractor (CSPDarknet53 software module), a feature combinator (C2f module) and an output predictive block (decoupled head module). YOLOv8 was developed by Ultralytics and had open source code available for further modernization to meet the needs of a specific project.

The hardware basis of the platform was a flying mobile system with a modular architecture (Fig. 1)

The authors proposed their own design of an unmanned aerial vehicle (quadcopter), assembled from the components manufactured in China and available for purchase from the distributors of such equipment.

To position UAVs relative to base stations and so-called anchors, the DWM1000 sensor was used — TWR positioning support (Two-Way Ranging, a method used to determine the distance between two devices using signal exchange) and the TDOA sensor (Time Difference of Arrival, a positioning method that determines the location of an object by measuring the difference in the time of arrival of its signal by several receiving stations (anchors) with known coordinates).

The sensor complex included three key types of sensors:

– spatial sensing equipment (Livox MID-40 multi-lens lidar, Orbbec Gemini 2 depth camera);

– visual perception equipment (DJI O3 Air Unit— a digital FPV system that included a compact 4K RGB camera and a transmitter);

– positioning and stabilization systems (DWM1000 sensors, NAV40 inertial modules, short-range detection systems based on ArduPilot firmware, backup navigation methods — visual odometry and optical navigation based on lidar and cameras).

The computing subsystem was based on the Aocoda-RC F765 V2 processor module with graphics acceleration, which supported the computer vision algorithms, neural network processing and data fusion.

The survey procedure for the above-ground crane runway of the supporting structure using an unmanned aerial vehicle was conducted in accordance with the following algorithm (Fig. 2):

1. Checking the condition and installing the charged battery of the UAV.
2. Calibration of the means of spatial and visual perception.
3. Performing a test flight to verify the operation of the navigation systems and the systems for capturing and transmitting point clouds of the 3d model.

4. Performing the initial scan (Fig. 3, pos. 1) to create a complete three-dimensional point cloud covering runways, crane beams, supports, with reference to the locations of the crane beam supports along the axes of both rails. At the same time, to ensure the integrity of the model, the scan could cover the floor of the room or the structural element connecting the crane beams of the A and B axes. The span distance was about one meter from the scanned elements. The speed was 5 m/s. The density of scan points was 2,000 pcs/m².
5. Performing an accurate scan of the runways of A and B axes to add a three-dimensional cloud of high-density points (Fig. 3, pos. 2). In this case, the flight was made at a distance of approximately 0.2 meters from the scanned elements. The speed was 1 m/s. The density of scanning points was 40,000 pcs/m².

6. Creation of a 3D model from the captured point cloud in the Geomagic Wrap application.
7. 3D model processing using artificial intelligence. At the same time, the following elements of the model were determined: rails, crane beams, supports, and lines locations. The processing was performed using a proprietary application based on the upgraded YOLOv8 model.
8. Model determination of deviations in the size and planned height position of runways from the limit values of GOST R 56944-2016 “Cranes. Overhead Crane Railways. General Specifications”⁴.
9. Identification of local defects of crane runways on the model.

In paragraphs 8 and 9 of the algorithm, the accuracy of the detected deviations and defects at the test stage of the method was verified by comparison with the results of visual and measuring control. After completing the tests of the method, the parameters of its accuracy would be indicated in the regulatory and technical documentation.

Research Results. A method of flaw detection of overhead crane runways using unmanned aerial vehicles has been developed. Local defects of crane runways have been identified. The data were obtained during the survey of overhead cranes as part of an expert survey of lifting cranes at machine-building enterprises.

In particular, the following tasks set by the authors were solved:

1. A three-dimensional point cloud was created (Fig. 4), which included rails, crane beams, supports, with reference to the locations of the crane beam supports along the axes of both rails (A-axis rail and B-axis rail). At the same time, the model provided a comprehensive view of the relative position of the path components along the A and B axes.

Figure 4 demonstrates a model of an overhead beam crane with crane runways obtained from a three-dimensional point cloud, the axes of A and B rails are indicated, as well as the line numbers — columns with a median line between them.

2. A three-dimensional cloud of high-density points was obtained for the rails of axis A and axis B in order to identify deviations of crane runway elements from the design position in plan and profile (compilation survey), as well as to detect defects in the crane runways (Fig. 5–10).

The photographs of the defects shown in Figures 5–10 were taken in the workshops of an industrial company during a real-life survey, using the available lighting and optical clarity of the air.

The defects were assessed using a trained computer vision neural network.

Figure 5 shows a defect in the joint of an I-beam with poor-quality welding of the fixing plate.

Figure 6 demonstrates a defect in the perforation of the I-beam of a suspended overhead crane (in the weld area).

Figure 7 shows an example of rail wear with chipping of the side edges and holes on the work surface.

Figure 8 provides the result of abrasion of the I-beam by the wheel of an overhead crane. The defect could occur as a result of bending of the beam or malfunction of the crane structures.

Figure 9 shows the wear of the rail as a result of wheel slipping of the support crane.

Figure 10 shows a defect in the thermic cut of the I-beam without a bottom plate.

3. The created three-dimensional model of crane runways allows for an automated assessment of deviations in the geometry of crane runways, wear and local defects with reference to the column sections and axes (Fig. 4).

Evaluation of the geometry of crane runways, presented as a model in Figure 4, gave the following results, which are shown on Figure 11.

The permissible difference in rail elevation within one cross-section along the axis of symmetry [P1] was 40 mm, and the permissible difference between adjacent columns [P2] was 10 mm (according to GOST R 56944–2016⁵). By comparing the data in Figure 11 to the allowable values, we could conclude that the geometry deviations in the crane track under consideration were within tolerance.

This method of flaw detection allowed us to obtain photo and video data, which could be used to estimate the size of defects with accuracy of one millimeter.

Discussion. The results obtained during the application of the method developed by the authors fully meet both the research goals and the objectives of its implementation. Achieving accurate and precise indicators was made possible by the large number of above-ground crane runway surveys, which allowed us to collect a variety of samples of defects in the geometry of crane runways and local defects. Data processing made it possible to train neural networks for computer vision and create fully functional three-dimensional models that can automatically identify and evaluate defects.

Comparison of the results obtained with literature data, theories, and other developments allows us to conclude that this study is unique. No such method has been developed before for overhead crane runways.

At the moment, the main disadvantage of the developed method is the limited operation time of an unmanned aerial vehicle due to a lack of battery power. It is not possible to install larger batteries, as this would require increasing the size of the unmanned aerial vehicle. Therefore, the maximum size of a drone can be no more than 0.5 meters, due to the limited space in enclosed areas, and the flight time can be no longer than 20 minutes (depending on the weight of the equipment).

The results of the study should be interpreted as positive. The method should be developed and implemented to improve the efficiency of inspection of overhead crane runways. In practice, the method allows you to:

— inspect crane runways from a close distance along their entire length;

— estimate the size of local defects using photo and video recordings and specialized software;

— create a 3D model of the crane runway using photogrammetry and assessing deviations from straightness in both the longitudinal and transverse planes;

— reduce labor intensity and duration of inspections by at least two times.

Conclusion. In conclusion, it should be noted that the work done by the authors has produced significant results, the most important of which is the development of a method for remote inspection of overhead crane runways. During the study, neural networks were trained to detect defects and an algorithm was created for examining crane runways by creating a three-dimensional model that allows for estimating geometric deviations in the rails in the longitudinal and transverse planes. The main advantage of this method is that it ensures the safety of experts and specialists who inspect lifting facilities. Additionally, the method enables the inspection of hard-to-access areas of crane runways and the identification of defects that may have gone unnoticed, reducing the likelihood of accidents.

Future research plans for lifting cranes involve the development of additional features to the system that will help identify their condition. The system will automatically inspect the crane runways and analyze the condition of the metal structures, identifying any defects.