Introduction

Home ladders are common and extremely useful in our daily activities. From making simple home repairs to accessing hard-to-reach places. It follows that ladders are an integral part of our lives. For a long time, research entities have been interested in ways to measure and evaluate the safety of ladder construction and workmanship. Examples include studies of damage1,2, conditions of ladder use3,4, or causes of falls5. In complex assembly processes, visual inspections, such as identifying scratches on work surfaces, identifying and selecting parts, etc., are crucial. They ensure product quality and provide information on the accuracy of the manufacturing process. Until now, inspections have been performed primarily by humans6, but recently they have been aided by artificial perception provided by computer vision systems (CVS)7. The successive reduction of production time has generated a demand for the systems for precise quality assessment of products. This is particularly important for many features that need to be checked in a short period of time. A human being, due to his limitations, becomes more unreliable at this point than, for example, a vision system. Perception and focus are strongly correlated with mood and fatigue. In the article8, the authors mention the influence of ergonomic factors on human reliability. In this regard, a study of the use of an eye tracking device to assess the performance of an assembly system from the perspective of an assembly system is interesting9. In this case, the results showed that simple and in-expensive solutions, such as changing the lighting system, can have an impact on quality-related costs. The article10 describes an inspection system for measuring the dimensions of electromechanical parts. It consists of two cameras located on the top and side. The system operates online without introducing delays in production. The image processing time was 300 ms. It is efficient, meets quality control requirements, and has contributed to waste and cost reduction. The article11 presents the use of deep learning-based methods to support visual inspection tasks on an automotive production line. Object detection was shown to be useful for assessing process quality. The introduction of various methods allows the evaluation of both processes and product quality without disrupting the production cycle. The disadvantage of deep learning systems is the high demand for computing power. In the article12, a machine vision system is proposed for precise and inexpensive geometric inspection of assembly. The system has been adapted for catalytic converter assembly and is based on algorithms and images of exhaust interfaces. An image segmentation procedure and a geometric model are presented for defect detection. The second article13 by these authors presented the development of a system that detects the flat and rotational displacement of the welded flanges relative to the ideal positions with a maximum error of less than one millimetre and one degree, respectively. Again, it was demonstrated that with the use of a vision system, it is possible to perform automatic quality control in real time without affecting the production time of the existing manufacturing process. There are also articles showing the results of research on checking the quality of manufactured elements using low-cost cameras. In the article14, the authors show the possibility of checking the quality of painted elements using an RGB-D camera. They demonstrate a recognition accuracy that can reach 99.32%. The article15 presented the use of commercially available line scanners from Keyence, working with small robots. The result was a fully automated quality control of the finished product. However, it should be borne in mind that this type of solution will be effective for the measurement of geometric quantities. The focus of this article is on domestic ladder solutions. However, it should be kept in mind that there are also special-purpose ladders16. However, regardless of the application when ladders are used, it is important to guarantee their completeness at the manufacturing stage. This is a key factor that affects the safety of use. The completeness of a ladder refers to whether all its components are present and in good condition. If one component is missing or damaged, it can lead to dangerous situations and accidents. Therefore, it is important to regularly check the condition of the ladder and ensure that all its parts are fully functional. Modern household ladders are usually made of durable materials that provide strength and stability. The most common choice is stainless steel sections, due to their strength and corrosion resistance. Materials also used include aluminium and high-quality plastics. Aluminium is a popular choice for its light weight and ease of handling, while stainless steel is credited with exceptional strength properties. Using a ladder made of the right materials and keeping it complete is crucial for user safety. The ladder should be regularly inspected for visible damage, such as cracks, deformation or wear. In addition, it should also be made sure that all joints are sufficiently strong and without signs of loose fitting. Standards for household ladders are an important part of ensuring safe use. Many countries and regions have regulations and guidelines for the design, manufacture, and use of ladders that aim to minimise the risk of accidents. An example of such a standard is the European Standard17, which specifies technical requirements for ladders used in homes and households. This standard covers aspects such as material strength, construction, load tests, and ladder marking. Compliance with these standards is important to ensure that ladders meet certain safety requirements. In addition to the standard17, there are a number of other norms for ladders that are used in different countries and regions. The EN 131 standard applies to all ladders in the European Union. Here are five examples of these:

  • 18: The American National Institute of Safety and Health (ANSI) standard specifies requirements for utility ladders. This standard includes specifications for strength, construction, testing, marking and operating instructions.

  • 19: The British National Standard (British Standard) standard for utility ladders. It specifies requirements for construction, strength, testing and marking. It is a frequently used standard in the UK.

  • 20: Australian-New Zealand standard regulating utility ladders. Includes requirements for design, materials, strength, testing and marking. It is used in Australia and New Zealand.

  • 21: Canadian Standards Association (CSA) standard for utility ladders. Specifies requirements for ladder construction, materials, testing and marking. It is a widely used standard in Canada.

  • 22: Japanese Standard (Japanese Industrial Standard) for ladders. Includes requirements for materials, construction, testing, marking and test methods. It is used in Japan.

Compliance with these standards is important to ensure a high level of safety when using household ladders. Introducing standardisation and meeting the requirements of these standards contributes to minimising the risk of accidents and ensuring the sound quality of ladders for users. Concern for safety is also manifested in the presence of institutions such as the American Ladder Institute23. Due to the length of domestic ladders and the need to accurately monitor their completeness, it is necessary to use specialised equipment for analysis and measurement. One of them is a line camera characterised by high resolution and scanning speed24,25,26. This enables a precise analysis of objects based on images. A line camera is a device with a narrow, rectangular array that captures images as lines of pixels. Thus, when scanning a ladder, a line camera can generate images with high horizontal resolution, which enables an accurate examination of each component of the ladder. Many manufacturers offer advanced software to analyse images and detect damage, such as cracks or deformations. In addition to a line camera, noncontact measurement systems such as GOM Pontos27,28,29,30 can be used to analyse the displacement of individual ladder elements. The GOM Pontos system is an advanced technology based on optical scanning that allows for precise measurement of the displacement of individual structural components. Thanks to advanced algorithms and accurate sensors, the GOM Pontos system is capable of detecting even the smallest movements and deformations of the ladder. Measuring the displacement of ladder components using the GOM Pontos system provides important data that can be used to analyse and evaluate the condition of the structure during operation (loading). The information obtained from the tests can allow appropriate corrective measures to be taken. With these advanced technologies, it is possible to meticulously monitor the ladder and identify deviations from the standard, which contributes to the safety of use and quality of manufactured products. This paper focusses on the development of a vision system to continuously analyse the completeness of ladders that come off the production line and to selectively check their condition by measuring deformation due to use as intended. The goal is to raise awareness of completeness of the the importance of ladder and introduce technological solutions that can contribute to even greater safety in use. The simultaneous use of line cameras and the GOM Pontos system provides an innovative solution for rapid analysis of the completeness of manufactured ladders. Thanks to these advanced technologies, it is possible to perform precise monitoring of finished products.

Methods and results

Metalkas S.A. company has three production plants with a total usable area of 42,287 \(m^2\) and offers over 2,000 different products, including aluminum ladders. The company employs over 550 employees and sells products to over 30 countries. The company within which the research was conducted is engaged in the production of ladders from prefabricated aluminium profiles. Currently, quality control was carried out individually by an employee. As a result of the need to check the quality of finished products, two workstations were proposed (Fig. 1). The first is ongoing testing of the completeness of ladder components, and the second is selective testing of the ladder. Consequently, the research was divided into two separate parts. In the first, the authors designed and constructed a chamber with cameras. This chamber can be installed on the production line according to the scheme in Fig. 1. All manufactured ladders pass through its program. The second part of the research focusses on selective offline testing with the GOM Pontos system. Selective testing involves taking a product randomly from a conveyor belt and conducting quality tests on it. Within the scope of this article, the focus is on measuring the displacements of notable elements of the ladder when it is used as intended. The authors examined how these displacements change as a function of the weight of a user.

Figure 1
figure 1

Scheme of the structure of the ladder examination system.

Full size image

A photo of the section of the production line where the initial tests were carried out is shown in Fig. 2. It presents a linear conveyor (from Fig. 1) with cameras mounted on tripods.

Figure 2
figure 2

Picture of an aluminium ladder production line during initial testing with cameras on tripods.

Full size image
Figure 3
figure 3

Ladders used in the study.

Full size image

The research was carried out on four-step and five-step domestic ladders. Figure 3 shows the basic types of ladder that has been checked. It was decided to verify that the tested ladders have such elements as

  • hook;

  • crossbars;

  • a rivet that allows the ladder to be opened;

  • side sticker;

  • platform supports;

  • plastic feet;

  • all steps.

Ladder completion check

The tests on this bench were aimed at checking the completeness (correct assembly) of the components that make up the ladder. The workstation to measure ladders moving on a conveyor belt consists of a conveyor belt to which two or three blackout chambers are attached. It is shown in Fig. 4, along with a view of the camera image.

The first chamber is equipped with two cameras. One of them is placed above the conveyor belt, and the other is placed from the side at a suitable angle (Fig. 5). The second chamber is equipped with a camera that looks at the other side of the ladder. The side cameras are placed between bands of rollers. To protect delicate optics, it is recommended to withdraw the lens beyond the face of the rollers (Fig. 5). The switching of individual measurement systems inside the chambers can be implemented by placing a sensor in front of each chamber to detect the presence of the ladder. It is also suggested to use vibration isolators in profile design to avoid the transmission of vibrations from the conveyor drive and actuators to the cameras. Measurement of the side of the ladder requires it to be orientated parallel to the conveyor belt (Fig. 6). This is necessary to ensure the perpendicularity of the lens axis with the ladder. The orientation procedure for scanning the ladder on the conveyor is shown in Fig. 6. Due to the fact that ladders are much longer than they are wide, the stand included 3 line cameras CA-HL02MX from Keyence. The camera for examining the ladder from above was equipped with a CA-LHE12 lens. The two cameras that examined the side edges were equipped with a CA-LHE16 lens. In addition, 3 light sources were used to provide even illumination of the ladder from above (CA-DBW50H and CA-DZW50X) and from the side (CA-DZW30X). The whole system was connected to an XG-X2900 controller along with CA-E200L, CA-DC60E and CA-DC40E extensions. A prototype station was designed and built to analyse the completeness of a manufactured ladder (Fig. 5). The stand was constructed using aluminium profiles. Two cameras were used during the investigation. The scanning of the second edge of a ladder was omitted to simplify the process of testing. The research aimed at verifying the applicability of the proposed solution. The second edge is identical to the first edge. Thus, it was possible to conduct tests with only one chamber.

Figure 4
figure 4

Scheme of a station with a conveyor belt and two measurement chambers.

Full size image
Figure 5
figure 5

Completeness test stand for prototype ladder assembly.

Full size image

The measurement procedure is presented in Fig. 6. In the first stage, the ladder is placed parallel to the wall of the measurement chamber, on which the side camera is placed. The conveyor belt drive is engaged. This causes the ladder to move through the measurement chamber area. The rotation of the motor shaft is coupled to an encoder that triggers the capture of images. The result is the movement of the ladder at a uniform feed rate according to the stages marked as two and three. The measurement ends at stage four, when the ladder is fully scanned by the side and top cameras. The ladder passed through the chamber positioned perpendicular to the camera’s lateral axis. This procedure ensured that the distance from the camera to the ladder was constant. Sharp images were obtained in such a way.

Figure 6
figure 6

Indication of the various stages of the measurement procedure: (a) start of taking a photo, (b) passage of the platform through the area of taking photos, (c) passage of steps through the area of taking photos, (d) end of taking a photo.

Full size image

Results of ladder completion check

Depending on whether all elements were found correctly in the image, the program reported two states. The first was when all elements were found correctly and the second was when at least one of the key elements was not found. The states were marked with a message and a colour. The camera took a photo of a single line of pixels every 5 pulses from the encoder coupled to the conveyor. The finished image is formed from the combination of 16,000 such lines. Figure 7 shows images of the ladder taken with line cameras.

Figure 7
figure 7

Example image of the scanned ladder.

Full size image

To analyse the completeness of a manufactured ladder, a dedicated program is needed to look for the right patterns in the recorded image. This program, after appropriate adjustment, was used in all the tests of individual ladders presented. Figure 8 contains a summary tree of the blocks of operations that the controller performs during the analysis. In the expansion of each block are the corresponding settings that affect the detected features. 100 ladder passes through the system were carried out. The correct recognition was obtained at the level of 99.9% of passes. The photo taken in this way was automatically analysed, according to a previously prepared program. Figure 9 shows correctly installed ladder components.

Figure 8
figure 8

Developed program to analyse completion of the ladder.

Full size image
Figure 9
figure 9

Specification of the elements subject to the ladder completion of the ladder.

Full size image

To verify that the system also detects missing ladder components, modifications were made to the ladder by removing, painting over, or covering up certain ladder components. Figure 10 shows a misplaced ladder. Fifty passes were conducted through the measurement system. All passes were correctly marked by the system as incorrect.

Figure 10
figure 10

Detection of errors in the assembly of individual components.

Full size image

Additionally, the effect of the colour of individual elements on the result was checked (Fig. 11). Below are the correctly detected elements in different colours. In the next step, an analysis of the effect of colour on the performance of the program was carried out. Images of ladders with elements in yellow, green, black, and red were analysed. In all cases, the system properly interpreted the results and identified all ladders as correct.

Figure 11
figure 11

Recognition of ladder feet of various colours.

Full size image
Figure 12
figure 12

Recognition of accessory ladder components, such as the manual, tool holder, and straps connecting the ladder arms.

Full size image

In the course of carrying out further research work, a four-step ladder was analysed. This time also includes an instruction manual wrapped around the third step and a tool holder attached with a clamp to the ladder platform. It is shown in Fig. 12. In addition to the previously mentioned items, it was decided to check whether the tested ladder has such elements as:

  • manual;

  • tool holder;

  • straps connecting the ladder arms.

Selective ladder tests

Additional selected ladders undergo selective testing. Every 1 h or when retooling the production line to a different type of ladder the test was being performed. The purpose of the research was to assess stiffness and strength dependent on weight of the user by measuring the deformation of the ladder during its operation. The proposed research was carried out using the GOM Pontos vision system. It is a two-camera system with integrated illuminators (Fig. 13). Thanks to the offset cameras and the angle between them, it is possible to calculate the position of any point. Special markers are placed on the ladder that are detected by the system. The view from two cameras on the same test item is shown in Fig. 7.

Figure 13
figure 13

GOM Pontos system and the view from its cameras.

Full size image

The created program tracked the position of the points marked on the image. The points were defined as shown in Fig. 14. Their displacement was measured in the XY plane. The recorded experiment consisted of a single ascent and descent from a ladder. Five attempts were made by three people of different weights (70, 90, 110 kg). Each time the ascent of the ladder was initiated with the right foot in the first step.

Figure 14
figure 14

Marking of measurement points and coordinate axes.

Full size image

The selection of research participants was dictated by the maximum permissible weight of the user of domestic ladders. In this case, the value was 120 kg and 150 kg. People with the greatest possible weight variation were selected to participate in the study. The assumed criterion was the mass of participants in the experiment who were also involved in technical aspects of project realization.

Results of selective ladder tests

Five measurements were taken for each of the three persons. This gives a total of 15 measurements of ascending and descending the ladder. Measurements were made according to the procedure outlined in section “Results of ladder completion check”. Displacements were measured in the XY plane (Fig. 8). For an accessible representation of the results, the displacement of the Y-axis is shown as a function of the measurement step. On the other hand, are shown by colour (Fig. 15). During the analysis of the results, attention was drawn to the very low displacement values of points a. They were distributed on an arm equipped with ladder steps. In Fig. 15, an additional preview of the displacements of the a points was provided, scaling the X axis from − 1 to 1 mm. Due to the small differences in the readings, further analysis focused only on points b.

Figure 15
figure 15

Visualisation of displacements on the Y and X axes (colormap) as a function of the measurement step, while ascending and descending the ladder.

Full size image

In Fig. 16, two sample cases of three individuals ascending and descending the ladder, with weights of 70, 90, and 110 kg, are indicated. The graphs are presented as visualizations of displacements on the Y and X axes (colormap) in relation to the measurement step during ladder ascent and descent. The horizontal green line represents the initial position of each point, indicating the degree of deviation in measurements. The obtained results demonstrate the individual nature of ladder usage for each user during ascent and descent, characterized by significant deviations from the initial position. The readings between the ascent and descent phases indicate the displacement of ladder components resulting from the user’s stance on the ladder platform. Notably, values in this case differ from those when the ladder is unburdened (visible at the end of each measurement—step> 90). Larger displacements are observed with increased user weight. Ladder descent does not equate to returning to the starting position, as ladder elements shift due to user load. Multiple consecutive measurements by the same user lead to progressively smaller displacements. Ladder components settle into a stable position after several ascents and descents, resulting in reduced displacement during subsequent measurement points. Resetting this state is achieved through ladder folding and unfolding.

Figure 16
figure 16

Visualisation of displacements on the Y and X axes (colormap) as a function of the measurement step, while ascending and descending the ladder.

Full size image

In order to increase the readability of the measurements, the mean and median values of the displacement of each point were calculated. Figures 17 and 18 illustrate these with the range of displacement values obtained during the presented samples. Figure 17 shows the displacements in the X axis. While in Fig. 17 are the displacements in the Y axis. It was expected to obtain a wedge shape, which is characterized by successively decreasing values as the notation of the measurement point increases. The points were marked along the length of the ladder arm in order from bottom to top. The highest displacement values are therefore obtained for the point located lowest. As can be seen from the figures, the points in the X axis tend to increase in value. There is therefore an upward displacement. In the Y axis, they decrease their values, which means that they move in the direction of the person using the ladder. The b points are set on the ladder arm, which is lacking steps. Thus, this is the part that stabilizes the position of the ladder. The results show that when using the ladder, the stabilizing arm moves toward the arm with steps, reducing the distance between them.

Figure 17
figure 17

The spread of displacement of individual points in the X axis with an indication of the mean and median values.

Full size image
Figure 18
figure 18

The spread of displacement of individual points on the Y axis with an indication of the mean and median values.

Full size image

The displacement values of individual points increase with the weight of the user. The results obtained in five trials for each user indicate that the spread of displacement values obtained is repeatable. However, this is not clear from the average values. For example, Sample 1 for a 90kg person has comparable average values with Sample 2 of a 110kg user. The differences are the spread of measurement readings, which is higher at 110kg. The users taking part in the study tried to use the ladder as reproducible as possible. However, this is very difficult to achieve.

Discussion

In this article, the authors present the application of an innovative workstation to analyse the completeness of manufactured ladders. The developed solution focusses on the use of advanced technology to accurately assess the correctness of the ladder assembly and identify possible defects. The modern approach to quality control in the ladder manufacturing process has the potential to bring significant benefits to manufacturers by improving product quality, increasing production efficiency, and reducing costs associated with the production of defective units. The developed stand consists of three main components: a camera, a computer, and specialised software. The analysis process begins with taking pictures of the ladder using the cameras. The resulting photo is transferred to the computer where the algorithms analyze the image and identify the various components of the ladder. If the analysis detects a missing or damaged component, the computer automatically generates a warning for the assembly line staff. The entire analysis process takes just over 1 s (1.04 s), making the workstation fast and efficient. The tests conducted on the stand confirm that its use on the ladder production line is highly effective. The ability to clearly determine whether all components of the ladder have been installed correctly is a key aspect of quality control. This gives the manufacturer confidence that the ladders that leave the production line are safe and meet all quality standards. A stand of this type can bring many benefits to manufacturers:

  • Quality improvement: The system makes it possible to accurately determine whether a ladder has been correctly assembled. This eliminates the risk of defective products, resulting in increased safety and reliability.

  • Increased efficiency: Due to a rapid analysis, the workstation makes it possible to detect defective ladders early in the process, allowing them to be eliminated quickly and avoid wasting time and resources on further production.

  • Cost reduction: Avoiding the production of defective ladders contributes to a significant reduction in costs associated with repairs, replacements, and complaints.

In second station, the tests were carried out on the GOM Pontos system, using advanced measurement technology and image analysis. For the tests, people of different weights, ranging from 70 to 110 kg, were selected to climb ladders and generate dynamic loads. During the experiments, the displacement of points of the ladder structure was monitored, paying special attention to points near the feet. The results of the study clearly show the relationship between the weight of the person and the displacement of the ladder points. As the weight of the user increases, both the maximum and average displacements of the points of the structure increase. Particularly significant are the displacements of points located at the feet of the structure of the ladder, suggesting that the ladder maintains its rigidity and does not bend even under heavier loads. The findings have important implications for the safety and durability of ladders. The finding that ladders maintain their rigidity even under loads approaching 110 kg is an important support for manufacturers and users, confirming that the structures are safe for people of different weights. The tests conducted on the GOM Pontos system provide important information on the behaviour of ladders under different loads. The results confirm that the ladders are rigid enough for people with body weights in the 70–110 kg range. Displacements of structural points, especially at the feet of the ladder, are minimal, indicating the high strength and stability of the ladder profile. This research contributes to a better understanding of the behaviour of ladder structures in actual use and is an important contribution to improving product quality and safety.

The realization of the test station involved the time of the scientific project and took about a year. The station was designed and built in such a way as not to prolong the developed production process. For this purpose, a section of the production line was used to transport the finished product to the worker’s pickup point. The product at this point exits the safety cage area of the production line. A standard ladder leaves the line every 12 s. This is enough time to take a measurement and analyze it. Previously, checking the quality of the ladder was performed by two workers. With the measurement system proposed within this article, the measurement process is done automatically. A longer survey is a selective test procedure. It takes about 10 min and is performed about every 1 h or when retooling the production line to a different type of ladder.

Conclusions

The ladder completeness analysis station is an innovative and effective tool in the quality control of the ladder manufacturing process. Its advantages, such as improving product quality, increasing efficiency, and reducing costs, contribute to the competitiveness of manufacturers in the market. With an advanced technology-based approach, it is possible to provide ladder customers with safe, reliable, and high-quality products. This post opens up new perspectives in the field of quality control and automation of production processes, making a valuable contribution to the development of the industry. Modern technologies enable for precise strength testing and analysis of the behaviour of structures under dynamic loads. In this article, the results of research conducted on the GOM Pontos system were presented. They focused on analysing the effect of the weight of a person climbing ladders on the displacements of various points of the structure. The goal of the research was to understand to what extent ladders maintain their stiffness and strength depending on the weight of the user and what displacements occur at different points of the structure. Further work will include the development of the control of the fabrication of aluminum ladders. The results presented here are the outcome of the project, which finds its application in a real working factory. Another goal of the authors will be to study the detection of, among other things, surface defects and other imperfections resulting from manufacturing processes.