Fig 1 - uploaded by Guillaume Jacques Laurent
Content may be subject to copyright.
Binary phase encoding process. (a 1 ) Black; typical interference signal. Red; missing fringes used for binary encoding. (a 2 ) Binary form of signal (a 1 ). (a 3 ): corresponding sequence of black and white stripes. (b 1 ) LFSR binary sequence with n = 4 bits. (b 2 ): binary words formed by 4 successive bits. (b 3 ): corresponding code value. (b 4 ): corresponding linear position. (c 1 ): example of 2D encoded area. (c 2 ) corresponding 1D x encoding. (c 3 ) corresponding 1D y encoding. (c 4 ): elementary coding cell made of 3 × 3 periods. The missing dot at the upper left corner aims to remove π/2 rotation ambiguities.
Source publication
Computer vision is a convenient noncontact tool for position control and thus constitutes an attractive multidirectional alternative to widely used single-direction sensors. However, to meet actual industry requirements, vision-based measurement methods must be sufficiently robust to comply with industrial environments. This article explores the ro...
Contexts in source publication
Context 1
... pattern used ( Fig. 1.c 1 ) is made of an altered 2D periodic distribution of dots that results from the product of 1D lines and columns ( Fig. 1.c 2 -c 3 ). The periodic frame is aimed to allow accurate measurements of δ x and δ y through phase measurements whereas alterations produced by the missing lines and columns encrypt a binary code ensuring the ...
Context 2
... pattern used ( Fig. 1.c 1 ) is made of an altered 2D periodic distribution of dots that results from the product of 1D lines and columns ( Fig. 1.c 2 -c 3 ). The periodic frame is aimed to allow accurate measurements of δ x and δ y through phase measurements whereas alterations produced by the missing lines and columns encrypt a binary code ensuring the unambiguous identification of the line and column orders as necessary to identify parameters k x and k y . Thus x and y are obtained ...
Context 3
... of lines and columns; ii) to provide information redundancy beneficial to decoding robustness; and iii) to relate encoding features to phase information. For that purpose, binary values 1 and 0 are represented by sets of three periods and distinguished by the central period that is present when the bit is equal to 1 and absent otherwise ( Fig. 1.a). In this way, the binary code reduces the magnitude of the periodic frame spectrum but does not alter its phase that encodes the position of the lines and columns with respect to the image pixel frame. This point constitutes a crucial improvement in regard to several encoding methods proposed earlier and inducing non-linearities [34], ...
Context 4
... encryption of the line and column orders is based on linear feedback shift register sequences that require a single binary track whatever the number of bits considered ( Fig. 1.b) [34], [36]. In this encoding scheme, each word of n bits shares n − 1 bits with its immediate neighbors and the absolute position can be retrieved from any sequence of n consecutive bits as depicted in Fig. 1.b for n = 4. Combined with the choice to encode each bit with three periods, this technique allows the encoding of a spatial ...
Context 5
... orders is based on linear feedback shift register sequences that require a single binary track whatever the number of bits considered ( Fig. 1.b) [34], [36]. In this encoding scheme, each word of n bits shares n − 1 bits with its immediate neighbors and the absolute position can be retrieved from any sequence of n consecutive bits as depicted in Fig. 1.b for n = 4. Combined with the choice to encode each bit with three periods, this technique allows the encoding of a spatial length equal to 3 · (2 n − 1) periods from a sequence of 3 · (2 n + n − 2) bits whereas only 3n consecutive periods are required for the decoding of the current position. Therefore, the ratio between the ...
Context 6
... of the current position. Therefore, the ratio between the measurement range and the minimum field of observation is given by (2 n − 1)/n and increases with n. Then the proper choice of the effective values of the physical periods λ x and λ y and of the bit number n allows the design of encrypted patterns suitable to many application requirements. Fig. 1 summarizes the encoding procedure from the differentiation of bit values ( Fig. 1.a), to the encryption of position by means of LFSR sequences ( Fig. 1.b), to end with the resulting 2D pattern ( Fig. 1.c). Thus, any 2D position of coordinate (x, y) is represented by a coding cell ( Fig. 1.c 4 ) made of 3 × 3 dots of which central lines ...
Context 7
... minimum field of observation is given by (2 n − 1)/n and increases with n. Then the proper choice of the effective values of the physical periods λ x and λ y and of the bit number n allows the design of encrypted patterns suitable to many application requirements. Fig. 1 summarizes the encoding procedure from the differentiation of bit values ( Fig. 1.a), to the encryption of position by means of LFSR sequences ( Fig. 1.b), to end with the resulting 2D pattern ( Fig. 1.c). Thus, any 2D position of coordinate (x, y) is represented by a coding cell ( Fig. 1.c 4 ) made of 3 × 3 dots of which central lines and column encode the values of x and y respectively. The four corners remain the ...
Context 8
... n. Then the proper choice of the effective values of the physical periods λ x and λ y and of the bit number n allows the design of encrypted patterns suitable to many application requirements. Fig. 1 summarizes the encoding procedure from the differentiation of bit values ( Fig. 1.a), to the encryption of position by means of LFSR sequences ( Fig. 1.b), to end with the resulting 2D pattern ( Fig. 1.c). Thus, any 2D position of coordinate (x, y) is represented by a coding cell ( Fig. 1.c 4 ) made of 3 × 3 dots of which central lines and column encode the values of x and y respectively. The four corners remain the same allover the 2D pattern with one missing corner to avoid π/2 ...
Context 9
... of the physical periods λ x and λ y and of the bit number n allows the design of encrypted patterns suitable to many application requirements. Fig. 1 summarizes the encoding procedure from the differentiation of bit values ( Fig. 1.a), to the encryption of position by means of LFSR sequences ( Fig. 1.b), to end with the resulting 2D pattern ( Fig. 1.c). Thus, any 2D position of coordinate (x, y) is represented by a coding cell ( Fig. 1.c 4 ) made of 3 × 3 dots of which central lines and column encode the values of x and y respectively. The four corners remain the same allover the 2D pattern with one missing corner to avoid π/2 ambiguities in the in-plane angle. In this way, every ...
Context 10
... patterns suitable to many application requirements. Fig. 1 summarizes the encoding procedure from the differentiation of bit values ( Fig. 1.a), to the encryption of position by means of LFSR sequences ( Fig. 1.b), to end with the resulting 2D pattern ( Fig. 1.c). Thus, any 2D position of coordinate (x, y) is represented by a coding cell ( Fig. 1.c 4 ) made of 3 × 3 dots of which central lines and column encode the values of x and y respectively. The four corners remain the same allover the 2D pattern with one missing corner to avoid π/2 ambiguities in the in-plane angle. In this way, every coding cell provides the local values of both foreground and background intensities ...
Context 11
... in Fig. 2.d 1 -d 2 . These phase planes can be expressed by: Φ 1 (i, j) = a 1 ·i+b 1 ·j +c 1 and Φ 2 (i, j) = a 2 ·i+b 2 ·j +c 2 where i and j are the pixel coordinates counted from the image center. In practice, coefficients a 1 , b 1 , c 1 and a 2 , b 2 , c 2 are determined by least square fitting of corresponding unwrapped phase planes (Fig. 2.d 1 and 2.d 2 respectively). The least square identification of these parameters allows the accurate determination of phases Φ 1 (0, 0), Φ 2 (0, 0) that, modulo 2π, correspond to the expected phases φ x , φ y of the pattern point imaged exactly at the center of the recorded image. The choice of the least square method constitutes an efficient and appropriate way ...
Context 12
... represented in Fig. 5.b 2 this criterion of binary value determination leads to keep the decision threshold at the middle of the background and foreground intensities whatever their values in regard of the camera quantification range (Fig. 5.b 1 ). Finally, determining the position of V 1 and V 2 in the LFSR sequence provides to the phase constants k x and k y . In this way, the absolute position of the phase-encoded pattern which is imaged exactly at the central pixel of the camera is finally obtained with respect to eq. 1. Supplemental material ...
Context 13
... the outside and thus reduce the entrance of external dust. An inverted microscope (Olympus IX73) with a 10× lens objective (Olympus UPLFLN 10x, N.A. 0.30) is used for the optical imaging. ...
Context 14
... the encoded pattern is fixed underneath. The camera used for acquisition is an IDS (UI-3280CP-M-GL) set at 20 frames per second encoded on 12 bits (allowing 4096 levels of gray). Fig. 6 summarizes the implementation of the experiment. An exposure of 10.5 ms is chosen as it reveals the highest lighting dynamics to acquire the pattern images ( Fig. 7.a 1 ...
Context 15
... on images of 1024 × 1024 pixels with random position and orientation distributed among the whole pattern area. Then the image is progressively occluded by black circles with a radius randomly distributed between 20 and 75 pixels and localized at a random position in the image. The process is then continued until 100 % of occlusion is obtained. Fig. 10.a 1 ,a 2 show typical occluded images with rates of occlusion of 15 % and 65 % respectively. The image obtained at each step is processed and the retrieved position is compared to the expected one as used for image generation. The reconstructed position is considered valid if the decoded binary sequence fits the correct ...
Context 16
... protocol is described visually by a short movie clip in supplemental material 4. and was repeated 300 times to gather statistically relevant information. Fig. 10.b presents the rate of positioning success as a function of the percentage of image occlusion. On the basis of these 300 random trials, a 100 % success rate is observed up to 15 % of occlusion and the probability of correct detection is above 90 % up to 55 % of occlusion. Furthermore, this excellent level of performance against ...
Similar publications
Rotation motion in a three-dimensional physical world refers to an angular displacement of an object around a specific axis in $\mathbb{R}^3$. It is typically formulated as a non-linear and non-convex process due to the nonlinearity and nonconvexity of $\mathbb{SO}(3)$. However, this paper proposes a new perspective that the 3D rotation motion can...
Citations
... An additional limitation is that they provide the position in a referential external to the microscope, while the camera is the relevant referential. Another family of position-sensing techniques is based on computer microvision 20 of machine-readable patterned position scales [21][22][23][24][25][26][27][28][29][30] . In this approach, a micropatterned scale contains encoded position information, and its images are decoded by appropriate software into in-plane positions and orientations. ...
... Resolutions of less than 2 nm have been reported, which shows that the technique works in the super resolution regime. It has been used to perform in-plane position determination in various contexts, including robotics [23][24][25] , machine vision 30 , and microscopy 16,21,22 . Our group has developed a particular family of machine-readable patterned position scales and associated vision software, termed nanoGPS. ...
Investigations of the in-plane positioning capabilities of microscopes using machine-readable encoded patterned scales are presented. The scales have patterns that contain absolute position information, and adequate software accurately determines the in-plane position from the scale images captured by the microscope camera. This makes in-plane positioning experiments simple and fast. The scales and software used in this study are commercially available. We investigated different microscopy systems and found that positioning performance is a system issue that is not determined solely by stage performance. In some cases, our experiments revealed software or hardware glitches that limited the positioning performance, which we easily fixed. We have also shown that it is possible to investigate vibrations using this approach and quantify their impact on image blurring. This is, for example, useful for experimentally determining the settling time after a stage movement.
... An additional limitation is that they provide the position in a referential external to the microscope, while the camera is the relevant referential. Another family of position sensing technique is based on imaging a scale with engineered patterns and retrieving the position and orientation from the image [17][18][19][20][21][22] . It has been used to perform in-plane position determination in various contexts, including robotics 17 , machine vision 23 , and microscopy 13,24 . ...
Investigations of the in-plane positioning capabilities of microscopes using commercially available nanoGPS OxyO scales are presented. The scales have patterns that contain absolute position information, and nanoGPS software accurately determines the in-plane position from the scale images captured by the microscope camera. This makes in-plane positioning experiments simple and fast. We investigated different microscopy systems and found that positioning performance is a system issue that is not determined solely by the stage performance. In some cases, our experiments revealed software or hardware glitches that limited the positioning performance, which we easily fixed. We have also shown that it is possible to investigate vibrations using this approach and quantify their impact on image blurring. This is, for example, useful for experimentally determining the settling time after a stage movement.
... A considerable of research has been done on non-contact on measuring for industrial parts, including stereopsis, light curtain, and grating projection profilometry [11,12]. Stereopsis is a depth-sensing technique that takes advantage of the difference (or disparity) between two cameras [13]. ...
As a linear actuator, accurate dimension measurement is crucial to the transmission reliability and interchangeability of ball screws. However, most of the current approaches are ineligible for rapid ball screw in-situ inspections due to the installation condition requirement of the production line. In this research, a machine vision method is presented to achieve highly accurate measurements of crucial parameters (the center distance and raceway arcs) in ball screws using a curvature edge detection algorithm. To capture images of the immediate area surrounding the area of interest, a telecentric lens is used. Thereafter, the curvature-based edge detection algorithm is employed to extract the contours. The measurement location on the object is automatically chosen by using a shape-matching algorithm. Additionally, random noise is suppressed by using the multiple-measurement averaging technique. Based on the results of the experiments, it is concluded that the center distance and the two raceway arcs computed absolute errors are 0.0019 mm, 0.0055 mm, and 0.0059 mm, respectively.
... This decoding step involves the application of a local threshold to different phase maps of the pattern leading to the retrieval of the fringe counts k 1 , k 2 and quadrant number k 3 . In this fashion, an absolute and unambiguous pose measurement with a high sub-pixel resolution can be achieved for the in-plane case, and the measurement shows great robustness across different image alterations (André et al., 2021). This article extends this in-plane pose determination method to a full 6-DOF pose measurement through a deeper analysis of the spectral phase in the Fourier domain. ...
The retrieval of an observed object’s pose is an essential computer vision problem. The challenge arises in many different fields, among them robotics control, contactless metrology, or augmented reality. When the observed object shrinks from the macroscopic scale to the microscopic, pose estimation is further complicated by the weaker perspective of imaging macroscale lenses down to the quasi-orthographic projection inherent to microscope objectives. This paper tackles this issue of microscale pose estimation in two complementary steps that rely on the use of planar periodic targets. We first consider the orthographic projection case as a means of presenting the theory of the method and showing how the pose of periodic patterns can be directly retrieved from the Fourier frequency spectrum of a given image. We then address the perspective case with long focal lengths, in which the full six-degrees of freedom (6-DOF) pose can be retrieved without ambiguities by following the same theoretical background. In addition to theoretically justifying pose retrieval via Fourier analysis of acquired images, this paper demonstrates the method’s actual performance. Both simulations and experimentation are conducted to validate the method and confirm an experimental resolution lower than 1 / 1000 th of a pixel for translations. For orientation measurement, resolutions below 1 μ rad. for in-plane orientation, and below 100 μ rad. for off-axis orientations can be achieved. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
... These dots are separated by a constant distance defined by the period of this signal in this direction. This method was subsequently developed and improved to widen its measurement range by adding special encryption to the signal, which made it possible to extend the capabilities of the method, in particular in terms of measurement range and in terms of resolution [9][10][11]. Other works have used the same principle of these sights to achieve nanometric precision at the level of continuous parallel robots [12]. ...
The measurement based on the vision of periodic patterns by a remote camera is a more suitable solution for the control and characterization of micro robotic systems. In this way, we can take precise measurements without disturbing the measured objects and without the need for more devices and sensors that cause great confusion and clutter around the workspace. The technique of projection of the coded periodic patterns is perhaps effective and of good performance and especially in the case of a well calibrated system. Indeed, the question arises on the influence of the calibration on the results of measurement by this technique in terms of precision and uncertainty. Thus, the main purpose of this article is to study the influence of calibration on the measurement process, by deducing and calculating the committed error and the uncertainty rate. Based on an algorithm in MATLAB, we first create a coded periodic test pattern to which we apply a certain imposed displacement. Then, we project this pattern using a projection and reduction system on a digital camera which captures a photo and transmits it to a processing algorithm, which, in turn, calculates and deduces a value for the measured displacement. The repetition of this process occurs several times on the same imposed displacement (the same standard) to provide us with several measured values, this redundancy of the data allows us to draw a calibration curve and deduce several metrological parameters from our system of measure.
... To address these needs and to overcome the obstacles, this paper presents a novel type of fiducial marker that is based on the design of regular QR codes but embeds a periodic pattern in place of the binary code. Indeed, André et al. recently demonstrated that periodic patterns lead to improved performance with typical subpixel resolutions of one thousandth of a pixel corresponding to nm and µrad measurement resolutions with a 10x microscope [37], [38]. The design of the resulting High-Precision markers (named HP codes) is introduced in section II along with the study of influential parameters and the method to efficiently achieve the pose estimation. ...
... Indeed the white centers of dots correspond to a wrapped phase of 0 rad whereas the black middles between dots correspond to a wrapped phase of ±π rad. More details on this phase processing can be found in [37], [38]. ...
At small scales, automating robotic tasks such as assembly, force/displacement characterization, positioning, etc., appear to be particularly limited. This is due to the lack of sufficiently performing and easy-to-implement multi-degrees-of-freedom measurement systems able to measure the relative pose between micro-parts. In order to address this issue, a measurement method based on High-Precision fiducial markers (named HP code) is proposed. This measurement method combines a periodic pattern (providing high resolution by phase-based computation) with more regular QR codes (bringing versatile implementations and a quick detection). The design and method to efficiently locate these HP codes are presented in this paper. Experimental investigations demonstrate ultra-high resolution:
$2$
nm and
$5$
$\upmu$
rad along
$X,Y$
and
$\Theta$
respectively (i.e. one thousandth of a pixel typically). The method is designed to be scalable as well as self-calibrated and to provide high robustness and high versatility. Two typical challenging applications in the field of microrobotics are automated to demonstrate these disruptive performances and the easy-to-implement capability of the method: (1) the automated assembly of two micro-fluidic chips through visual servoing with an achieved positioning accuracy below
$50$
nm, and (2) the automated micromechanical characterization of single fibers achieved by the integration of HP codes into a compliant structure enabling simultaneous micro-force and displacement sensing capabilities. These achievements highlight the versatility of the method and open the door to the rapid automation of high-quality robotic tasks at the micro scale.
Note to Practitioners
—The motivation for this work/study is based on the fact that many application areas are extensively orienting towards microrobotic systems to perform precise tasks with versatility. However, at the micro scale, many disturbances such as the effects of climate change strongly affect this precision. This problem is amplified by the fact that sensors cannot be easily integrated, either by lack of space or by the lack of measurement systems available. Vision-based approaches are widespread at this scale and appear very promising to measure the relative pose between micro-parts. Nevertheless, existing vision-based approaches like digital image correlation are both scale and texture dependent. Due to the lack of space, they are also difficult to use in practice at small scales for high resolution measurement. The main contribution of this paper lies in the capability to achieve ultra-high resolution measurements. For that, a structure based on High-Precision fiducial markers (named HP codes) is proposed and requires few and simple settings while achieving very high resolution both in position and orientation, typically down to one thousandth of a pixel and a few micro radians, respectively. It provides an off-the-shelf solution, versatile, easy to implement and achieves high resolution measurements in the plane (XY
$\Theta$
). HP codes are applicable to a wide range of applications such as tracking of a component/part of a mobile or deformable system, visual servoing of microrobots, positioning of samples, assembly of components or even mechanical characterization. A free distribution of the library is available online at https://projects.femto-st.fr/ vernier/.
... Grâce à cette représentation de la mire en fonction des constantes de phase, nous avons montré comment effectuer un seuillage local assurant une binarisation robuste de la mire pour en extraire la partie de la séquence codée observée. Cette extension significative du rapport plage sur résolution apportée par la méthode décrite dans ce chapitre a fait l'objet d'une publication dans le journal Transactions on Instrumentation and Measurement, montrant ainsi l'apport de cette méthode en termes de plage de mesure et de robustesse aux altérations possibles des images en microscopie [André20a]. ...
La mesure de pose à l'échelle microscopique est un défi majeur avec de nombreuses applications en microrobotique. À l'échelle macroscopique, la vision par ordinateur permet de mesurer la pose 3D d'objets de manière précise et robuste. Cependant, à mesure que l'échelle se réduit, les problèmes inhérents à la microscopie, tels que la faible profondeur de champ, un champ de vision réduit ou encore la projection orthographique des objectifs de microscopes, impliquent une meilleure structuration de l'objet d'intérêt.Cette thèse aborde cette problématique en utilisant des mires à motifs périodiques. La redondance structurée d'information contenue dans celles-ci permet d'atteindre des résolutions de l'ordre du millième de pixel par la mesure de phase.Le problème inhérent de la plage de mesure réduite au champ de vision du microscope est résolu en insérant une séquence binaire, sans perturbation de phase pour conserver la résolution de la méthode. La mire pseudo-périodique résultante fournit ainsi une mesure sur une plage de 10^5 x 10^5 px².En utilisant un seuillage adaptatif local basé sur l'information de phase, la méthode démontre une grande robustesse au bruit, à l'occultation ainsi qu'aux défauts de mise au point, rendant ainsi la méthode adaptée pour un emploi en microrobotique.Cette méthode a été validée par plusieurs expériences démontrant une résolution planaire de 1 nm en translation sur une plage de 10 x 10 cm² et 4.5 µrad. en orientation sur une rotation de 2pi. La mesure des angles hors-plan atteint une résolution de 100 µrad. sur une plage de [-pi/8; pi/8] rad. Finalement, le mémoire présente l'utilisation de la méthode de mesure dans plusieurs applications de microrobotique où une mesure de pose résolue et sans contact est nécessaire.
... The figure is taken from (b) NanoGPS encoder. A scientific product from HORIBA, Ref. [129]. ...
... Computer vision offers an effective way to retrieve the absolute position of an object by searching for features. The features can be rectangle, grating lines, circles, or any other pseudoperiodic patterns [129], which can be fabricated on the main scale and served as a measurement standard. This kind of scale is always absolutely encoded, and is called "encoded target" or "patterned substrate". ...
Optical position encoders have been invented and investigated for several decades for precision measurement. In this article, a comprehensive survey on the optical position encoders, from two aspects of industrial products and academic instruments, is presented. The technical background of the optical position encoder is introduced, and the popularity is analyzed through comparison towards other high-accuracy systems. The classification is conducted together with the typical application summarization. The resolution and accuracy enhancement strategies, the fabrication and range enhancement methods, the calibration instruments, and the multi-axis designs of the optical position encoders are reviewed. The original work on the relationship between relative orientation and signal distortion has been presented. The potential trends for future researches of the optical position encoders are also suggested.
... Such sensing technique can take advantage of a dedicated encoding, so as to sense over a long range and to be able to decouple different axes. The encoding strategy defined in [23] and [24] demonstrate the possibility of a precise and multi-axis sensing capability using vision. Such encoding combining together with micro-sized patterning, can help to minimize the number of pattern needed in the field of view for sensing. ...
... The approach of sensing in the proposed work, consists of visual processing to retrieve data x CF V , z CF V and α which then need to be converted into force / torque data. This step is based on an encoded pattern of the type described in [23], [24] as well as suited decoding algorithms. Basically, the encoded pattern is made of a 2D periodic distribution of dots altered by missing lines and columns as shown in Fig. 4. The periodic frame allows fourier computations that result in the conversion of the pattern axes X and Z (position x and z respectively) into two linear phase maps (Φ x (x) and Φ z (z) respectively) defined by Eq. 1. ...
This paper presents a novel multi-axis Force sensor with a range-to-resolution ratio of 55000, which makes the sensing device highly useful for wide range of applications. The presented device relies on a sensing strategy where a 2D encoded micro-pattern is encrypted to the mobile part of the compliant platform. This encoded pattern allows multi-axis visual sensing of displacements over a long range whereas Fourier computations ensures sub-pixel interpolation leading to the high resolution. The device is fabricated on a silicon wafer by means of clean room technology to comply with both precision encoding requirements and reliable mechanical behavior. The work presents the design, modeling and fabrication as well as the experimental validation of the multi-axis Force- Torque sensor. A modeling strategy is proposed for an effective estimation of force/torque with its capability to adapt non-linear stiffness evolution during the external loading. Experimentation demonstrates a sensing resolution of 2 N over a validated range of 110 mN. The proposed sensor measures the force along the two planar axes, and also the torque along the axis orthogonal to the plane, thus validating its axial decoupling capabilities.
