Coverage area for standard and wide-angle lens cameras mounted on the rear of a typical SUV 

Contexts in source publication

Context 1

... automotive collisions, with 20% (or 1832 children) of these injuries caused by the vehicle moving backwards. Between 2001 and 2003, the CDC reported that an estimated 7475 children (2492 per year) were treated for vehicle back-over injuries [16]. Again, although the blind- zone is not directly implicated in these injuries, it is reasonable to assume that a significant proportion of these injuries were because of the children being present in the vehicle’s blind-zone. Wang and Knipling [17] estimated that lane change / merge crashes in 1991 accounted for approximately 244 000 police-reported crashes with 224 associated fatalities. Furthermore, the authors reported that the principal causal factor in such crashes is that the driver ‘did not see the other vehicle’. Because of the increasing awareness of the vulnerability of pedestrians, legislation has been introduced, or is in the process of being introduced, in several jurisdictions around the world. In this section, the legislative or potential requirements for the visibility of blind-zones in the EU, Japan and the US are described. In the EU, legislation in the form of Directive 2003 / 97 / EC [18] was introduced in 2003. Although the initial requirements of this directive aim to reduce collisions caused by the blind-zones of LGVs and improve road safety for new vehicles circulating from 2006 / 2007 onwards, the legislation does not cover the existing fleet of lorries in the EU. However, as it has been estimated that existing fleets will not be fully replaced until 2023, Directive 2007 / 38 / EC [19] was introduced in 2007. This legislation requires the retrofitting of the required indirect vision systems (IVSs) to all existing fleets within 24 months of enactment of the bill (i.e. by July 2009). The shaded areas in Fig. 3 show the areas of a left-hand drive LGV’s environment that must be visible to the driver via the use of IVS, as required by these directives. Although the figure shows a left-hand drive vehicle, for right-hand drive vehicles within the jurisdiction of the legislation, the areas of required coverage are reversed. Examples of IVSs include additional mirrors to the standard rear-view mirrors (internal and external), as well as camera– monitor devices. However, practical problems arise with the use of additional mirrors as the extra mirrors can themselves introduce additional blind-zones, by obstructing direct forward vision, as well as having additional cost and styling implications. There is a clear provision for the use of camera– monitor devices for the coverage of vehicle blind-zones in this directive. In fact, the use of camera– monitor devices over mirrors is often not only desirable, but also necessary in certain situations. For example, it is practically impossible to cover the area at the rear of an LGV with mirrors alone, and so camera– monitor systems are the only practical solution. In Japan, legislation has also been proposed, which would require medium and large vehicles to be equipped with devices that allow the driver to detect objects in the vehicle’s blind-zones, either directly or indirectly using mirrors or camera– monitor devices (Fig. 4) [20]. For the purpose of the proposed legislation, a cylinder 1 m high with a diameter of 0.3 m placed anywhere within the coverage areas must be at least partially visible to the driver of the LGV directly, by mirror or by camera. However, in this legislation, it is proposed that objects within blind-zones caused by A-pillars and external mirrors need not be visible to the driver of the vehicle [7]. In the US, proposed legislation, in the form of S.694 (The Cameron Gulbransen Kids and Cars Safety Act of 2007) [21], is designed to protect against children being injured or killed in non-traffic incidents, such as when the vehicle is reversing. Relating to the potential use of cameras, the S.694 bill requires a ‘field of view (FOV) to enable the driver of a motor vehicle to view areas behind the motor vehicle to reduce death and injury resulting from backing incidents, particularly incidents involving small children and disabled persons’. Unlike the EU and Japanese legislation, however, the US legislation fails to describe in any technical detail the methods by which the objectives of the bill are to be implemented. As we shown in previous sections, the blind-zones on some vehicles can be quite large, particularly for vehicles such as SUVs and LGVs. The aim of this paper is to demonstrate that a good quality, undistorted image of a vehicle’s blind- zones can be displayed to the driver of the vehicle using a wide-angle camera system. As shown in Fig. 5 a , standard lens camera systems (e.g. 45 8 FOV lenses) are unable to fully cover the blind-zone of some SUVs. Considering that camera– monitor systems generally display a ‘reference’ point (i.e. part of the body of the vehicle) on screen, a standard lens camera with an FOV of 45 8 can only cover, perhaps, 1 m of the SUV blind-zone. Fig. 5 b illustrates how the use of a wide-angle lens camera system (e.g. . 100 8 FOV lenses) enables the entire SUV rearward blind-zone to be covered. Fig. 6 shows a sample placement of two wide-angle cameras mounted on an LGV. Camera 1 is a 135 8 wide- angle camera, located approximately half-way down the length of the LGV, and 3 m off the ground plane. The optical axis of camera 1 is tilted at 15 8 from the side of the LGV trailer. Camera 2 is a 135 8 wide-angle camera, located in the middle of the front cabin at about 2 m off the ground plane. The optical axis of camera 2 is tilted at 20 8 from the front face of the cabin. With both cameras corrected for distortion, Fig. 7 shows the areas in the vicinity of the vehicle that can be displayed to the driver. Such a camera system would cover all the blind zones of the LGV shown in Fig. 2, and would meet the requirements of both the EU Directive 2003 / 97 / EC (Fig. 3) and the proposed Japanese legislation (Fig. 4). Certain areas around LGVs need very wide wide-angle lens cameras (e.g. fish-eye lens cameras) to display the entire blind-zone to the driver. However, problems arise because of the deviation of wide-angle lens cameras from the rectilinear pin-hole camera model, because of geometric distortion effects caused by lens elements. Fish- eye cameras deviate substantially from the pin-hole model, introducing high levels of geometric nonlinear distortion. Because of this distorted representation of the real-world scene onscreen, there is potential for obstacles and VRUs to not be recognised by the driver. Additionally, the distortion may cause the driver to mis-judge the distance to objects, because of the nonlinearity of the view presented. Thus, camera calibration and distortion correction are important tasks for automotive camera applications. Not only do they make images captured by the camera more visually intuitive to the human observer, they are often also necessary for computer vision tasks that require the extraction of geometric information from a given scene. The following sections describe some of the effects of wide-angle and fish-eye lens distortion. Radial lens distortion causes points on the image plane in the wide-angle / fish-eye camera to be displaced in a nonlinear fashion from their ideal position in the rectilinear pin-hole camera model, along a radial axis from the centre of distortion in the image plane (Fig. 8). The visual effect of this displacement in fish-eye optics is that the image will a higher resolution in the foveal areas, with the resolution decreasing nonlinearly towards the peripheral areas of the image. For normal and narrow FOV cameras, the effects of radial distortion can be considered negligible for most applications. However, in wide-angle and fish-eye lenses, radial distortion can cause severe problems, not only visually but also for further processing in applications such as object detection, recognition and classification. Additionally, the radial distortion introduced by these lenses does not preserve the rectilinearity of an object in its transformation from real- world coordinates to the image plane. Straight lines in the real world can usually be approximated as circular sections in the distorted image plane [22 – 24], as evidenced in Fig. 9. The models described in this section are relationships between the distorted radial distance, r d , and the corresponding undistorted radial distance, r u . Both are measured from the COD (described in Section 3.1). Models of radial distortion can be grouped into two main categories: polynomial and non- polynomial models. The use of polynomials to model radial distortion in lenses is well established [25– 33]. From an embedded point of view, polynomials are desirable as they do not require the implementation of computationally demanding numerical algorithms, in contrast to log and tan-based functions that are required for non-polynomial models (however, through the use of look-up tables, this advantage is lessened). With the exception of the division model, the models described in this section are the functions of the undistorted radial distance, that is r d is a function of r u . It is usually necessary to convert a distorted image to an undistorted image, and thus obtaining r u as a function of r d is desirable. Problems arise with polynomial models because of the fact that there is no general analytical method to invert them, that is, there is no general method to invert a forward model to an inverse model for use in the radial distortion correction. However, back-mapping (described in Section 2.3.2) provides a means by which the forward model can be used to convert a distorted image to an undistorted rectilinear image. The standard model for radial distortion is an odd-ordered polynomial, as described by Slama in [25] and, subsequently, used in [26, 29, 32, ...

Context 2

... claim that 49.5% (or 466 children) of the fatalities were because of the vehicle reversing while children were present in the vehicle’s rearward blind-zone. In a study between July 2000 and June 2001, the Centers for Disease Control and Prevention (CDC) [15] reported that there were an estimated 9160 non-fatal injuries to children in non-traffic automotive collisions, with 20% (or 1832 children) of these injuries caused by the vehicle moving backwards. Between 2001 and 2003, the CDC reported that an estimated 7475 children (2492 per year) were treated for vehicle back-over injuries [16]. Again, although the blind- zone is not directly implicated in these injuries, it is reasonable to assume that a significant proportion of these injuries were because of the children being present in the vehicle’s blind-zone. Wang and Knipling [17] estimated that lane change / merge crashes in 1991 accounted for approximately 244 000 police-reported crashes with 224 associated fatalities. Furthermore, the authors reported that the principal causal factor in such crashes is that the driver ‘did not see the other vehicle’. Because of the increasing awareness of the vulnerability of pedestrians, legislation has been introduced, or is in the process of being introduced, in several jurisdictions around the world. In this section, the legislative or potential requirements for the visibility of blind-zones in the EU, Japan and the US are described. In the EU, legislation in the form of Directive 2003 / 97 / EC [18] was introduced in 2003. Although the initial requirements of this directive aim to reduce collisions caused by the blind-zones of LGVs and improve road safety for new vehicles circulating from 2006 / 2007 onwards, the legislation does not cover the existing fleet of lorries in the EU. However, as it has been estimated that existing fleets will not be fully replaced until 2023, Directive 2007 / 38 / EC [19] was introduced in 2007. This legislation requires the retrofitting of the required indirect vision systems (IVSs) to all existing fleets within 24 months of enactment of the bill (i.e. by July 2009). The shaded areas in Fig. 3 show the areas of a left-hand drive LGV’s environment that must be visible to the driver via the use of IVS, as required by these directives. Although the figure shows a left-hand drive vehicle, for right-hand drive vehicles within the jurisdiction of the legislation, the areas of required coverage are reversed. Examples of IVSs include additional mirrors to the standard rear-view mirrors (internal and external), as well as camera– monitor devices. However, practical problems arise with the use of additional mirrors as the extra mirrors can themselves introduce additional blind-zones, by obstructing direct forward vision, as well as having additional cost and styling implications. There is a clear provision for the use of camera– monitor devices for the coverage of vehicle blind-zones in this directive. In fact, the use of camera– monitor devices over mirrors is often not only desirable, but also necessary in certain situations. For example, it is practically impossible to cover the area at the rear of an LGV with mirrors alone, and so camera– monitor systems are the only practical solution. In Japan, legislation has also been proposed, which would require medium and large vehicles to be equipped with devices that allow the driver to detect objects in the vehicle’s blind-zones, either directly or indirectly using mirrors or camera– monitor devices (Fig. 4) [20]. For the purpose of the proposed legislation, a cylinder 1 m high with a diameter of 0.3 m placed anywhere within the coverage areas must be at least partially visible to the driver of the LGV directly, by mirror or by camera. However, in this legislation, it is proposed that objects within blind-zones caused by A-pillars and external mirrors need not be visible to the driver of the vehicle [7]. In the US, proposed legislation, in the form of S.694 (The Cameron Gulbransen Kids and Cars Safety Act of 2007) [21], is designed to protect against children being injured or killed in non-traffic incidents, such as when the vehicle is reversing. Relating to the potential use of cameras, the S.694 bill requires a ‘field of view (FOV) to enable the driver of a motor vehicle to view areas behind the motor vehicle to reduce death and injury resulting from backing incidents, particularly incidents involving small children and disabled persons’. Unlike the EU and Japanese legislation, however, the US legislation fails to describe in any technical detail the methods by which the objectives of the bill are to be implemented. As we shown in previous sections, the blind-zones on some vehicles can be quite large, particularly for vehicles such as SUVs and LGVs. The aim of this paper is to demonstrate that a good quality, undistorted image of a vehicle’s blind- zones can be displayed to the driver of the vehicle using a wide-angle camera system. As shown in Fig. 5 a , standard lens camera systems (e.g. 45 8 FOV lenses) are unable to fully cover the blind-zone of some SUVs. Considering that camera– monitor systems generally display a ‘reference’ point (i.e. part of the body of the vehicle) on screen, a standard lens camera with an FOV of 45 8 can only cover, perhaps, 1 m of the SUV blind-zone. Fig. 5 b illustrates how the use of a wide-angle lens camera system (e.g. . 100 8 FOV lenses) enables the entire SUV rearward blind-zone to be covered. Fig. 6 shows a sample placement of two wide-angle cameras mounted on an LGV. Camera 1 is a 135 8 wide- angle camera, located approximately half-way down the length of the LGV, and 3 m off the ground plane. The optical axis of camera 1 is tilted at 15 8 from the side of the LGV trailer. Camera 2 is a 135 8 wide-angle camera, located in the middle of the front cabin at about 2 m off the ground plane. The optical axis of camera 2 is tilted at 20 8 from the front face of the cabin. With both cameras corrected for distortion, Fig. 7 shows the areas in the vicinity of the vehicle that can be displayed to the driver. Such a camera system would cover all the blind zones of the LGV shown in Fig. 2, and would meet the requirements of both the EU Directive 2003 / 97 / EC (Fig. 3) and the proposed Japanese legislation (Fig. 4). Certain areas around LGVs need very wide wide-angle lens cameras (e.g. fish-eye lens cameras) to display the entire blind-zone to the driver. However, problems arise because of the deviation of wide-angle lens cameras from the rectilinear pin-hole camera model, because of geometric distortion effects caused by lens elements. Fish- eye cameras deviate substantially from the pin-hole model, introducing high levels of geometric nonlinear distortion. Because of this distorted representation of the real-world scene onscreen, there is potential for obstacles and VRUs to not be recognised by the driver. Additionally, the distortion may cause the driver to mis-judge the distance to objects, because of the nonlinearity of the view presented. Thus, camera calibration and distortion correction are important tasks for automotive camera applications. Not only do they make images captured by the camera more visually intuitive to the human observer, they are often also necessary for computer vision tasks that require the extraction of geometric information from a given scene. The following sections describe some of the effects of wide-angle and fish-eye lens distortion. Radial lens distortion causes points on the image plane in the wide-angle / fish-eye camera to be displaced in a nonlinear fashion from their ideal position in the rectilinear pin-hole camera model, along a radial axis from the centre of distortion in the image plane (Fig. 8). The visual effect of this displacement in fish-eye optics is that the image will a higher resolution in the foveal areas, with the resolution decreasing nonlinearly towards the peripheral areas of the image. For normal and narrow FOV cameras, the effects of radial distortion can be considered negligible for most applications. However, in wide-angle and fish-eye lenses, radial distortion can cause severe problems, not only visually but also for further processing in applications such as object detection, recognition and classification. Additionally, the radial distortion introduced by these lenses does not preserve the rectilinearity of an object in its transformation from real- world coordinates to the image plane. Straight lines in the real world can usually be approximated as circular sections in the distorted image plane [22 – 24], as evidenced in Fig. 9. The models described in this section are relationships between the distorted radial distance, r d , and the corresponding undistorted radial distance, r u . Both are measured from the COD (described in Section 3.1). Models of radial distortion can be grouped into two main categories: polynomial and non- polynomial models. The use of polynomials to model radial distortion in lenses is well established [25– 33]. From an embedded point of view, polynomials are desirable as they do not require the implementation of computationally demanding numerical algorithms, in contrast to log and tan-based functions that are required for non-polynomial models (however, through the use of look-up tables, this advantage is lessened). With the exception of the division model, the models described in this section are the functions of the undistorted radial distance, that is r d is a function of r u . It is usually necessary to convert a distorted image to an undistorted image, and thus obtaining r u as a function of r d is desirable. Problems arise with polynomial models because of the fact that there is no general analytical method to invert them, that is, there is no general method to invert a forward model to an inverse model for use in the radial distortion correction. However, back-mapping (described in Section 2.3.2) provides a means by ...