ArticlePDF Available

WRAPP-up: A Dual-Arm Robot for Intralogistics

Authors:

Abstract and Figures

The diffusion of the e-commerce has produced larger and larger volumes of different items to be handled in warehouses, with the effect to increase the need for picking automation. Conventionally, automation can be achieved through a custom plant in case of large scale productions where the items have well-known characteristics that are expected to change slowly and little over time. However, today the challenge is to realize a solution that is flexible enough to handle goods with different shapes, sizes, physical properties, and grasping modes. To solve this problem we first analyzed how humans perform picking and then synthesized their behavior in four main tactics. These have been used as guidelines for the design, the planning and the control of WRAPP-up: a dual arm robot composed of two anthropomorphic manipulators, a Pisa/IIT SoftHand and a Velvet Tray. The system has been validated and evaluated through extensive experimental tests.
Content may be subject to copyright.
WRAPP-up: a Dual-Arm Robot for Intralogistics
Manolo Garabini,+, Danilo Caporale,+, Vinicio Tincani ],,
Alessandro Palleschi,+, Chiara Gabellieri ,+, Marco Gugliotta, Alessandro Settimi,+,
Manuel G. Catalano ],, Giorgio Grioli],, Lucia Pallottino,+
Abstract The diffusion of the e-commerce has produced
larger and larger volumes of different items to be handled
in warehouses, with the effect to increase the need for pick-
ing automation. Conventionally, automation can be achieved
through a custom plant in case of large scale productions where
the items have well-known characteristics that are expected to
change slowly and little over time. However, today the challenge
is to realize a solution that is flexible enough to handle goods
with different shapes, sizes, physical properties, and grasping
modes. To solve this problem we first analyzed how humans
perform picking and then synthesized their behavior in four
main tactics. These have been used as guidelines for the design,
the planning and the control of WRAPP-up: a dual arm robot
composed of two anthropomorphic manipulators, a Pisa/IIT
SoftHand and a Velvet Tray. The system has been validated
and evaluated through extensive experimental tests.
I. INTRODUCTION
E-commerce, i.e., buying and selling physical goods via
services over the internet, has now reached his full devel-
opment. Led by Amazon, which accounted for more than
50% of the growth of the whole e-commerce market, and
by Alibaba, in 2017, retail e-commerce sales amounted to
more than 2 USD trillion with an annual growth rate higher
than 25% [1]. The expansion of e-commerce is affecting the
way warehouses work, especially the intralogistics, i.e., the
internal flow of goods within a distribution center [2].
On one side, the market growth led to an increase of the
employment: data from the Census Bureau [3] show that, in
the U.S., there was an annual growth rate in the Warehousing
and Storage (North American Industry Classification System
- NAIC - 493) employment of the 28% (from 2015 to 2016)
and that in 2016 the total workforce reached more than 600K
units. An analysis conducted by Data USA on the Census
Bureau ACS PUMS 1-Year Estimate data shows that material
movers are the largest share (20%) of jobs [4].
On the other side, a strong effort is devoted to maxi-
mize intralogistics efficiency by fully employing optimiza-
tion techniques [5], by pushing the productivity of human
operators even if it may cause high workloads [6], and by
adopting automated solutions. Finally, e-commerce impacted
both business-to-consumer (B2C) and business-to-business
(B2B) markets. The market size of B2B e-commerce is more
Centro di Ricerca “Enrico Piaggio”, Universit`
a di Pisa, Largo Lucio
Lazzarino 1, 56126 Pisa, Italy
+Dipartimento di Ingegneria dell’Informazione, Universit`
a di Pisa, Largo
Lucio Lazzarino 1, 56126 Pisa, Italy
]Soft Robotics for Human Cooperation and Rehabilitation, Fondazione
Istituto Italiano di Tecnologia, via Morego, 30, 16163 Genova, Italy
manolo.garabini@unipi.it
Fig. 1. WRAPP-up: a dual arm robot composed of two anthropo-
morphic 7-dof manipulators, a Pisa/IIT SoftHand, and a the Velvet
Tray. In the picture WRAPP-up is picking a box that does not have
the top surface.
than ten times the one of B2C [7], allowing to show an
unprecedented variety of products to the customers. This
brought undeniable advantages in terms of sales [7] but
also increased the flexibility requirements with which the
intralogistics system must comply.
Order picking - the process of retrieving products from
storage (or buffer areas) in response to a specific customer
request - is responsible for the 50-75% of the total costs
for a conventional warehouse [8]. Hence, order picking is
considered one of the highest priority areas to improve to
maximize a warehouse productivity. However, despite the
crucial importance of picking operations in warehouses, they
still mostly rely on human workers [9].
The major challenge preventing the full automation of
picking is represented by the high variability of objects to
handle in terms of shapes or object configurations, not ac-
cessible or even absent surfaces, flexible or pierced surfaces,
to name a few. Regarding object shapes, cuboids constitute
the vast majority of all the items stored in warehouses [10].
According to [11], among the shipped packages, the shapes
that occur more often are cuboids and, even if in a lower per-
centage, cylinders. Thus, strategies to manipulate cylinders
and cuboids in different configurations handle a considerably
large part of the goods in intralogistics processes.
Several are the components that concur to the realiza-
tion of a flexible picking solution: a robot design able to
execute the picking operations in a warehouse environment
physically; a vision system to detect objects and constraints;
a perception system able to identify desired and undesired
contacts; a planning method able to generate a trajectory
accomplishing the task while satisfying the constraints and
adapt it based on the perception outcomes; a control strategy
able to track the desired trajectory. This work is focused on
the realization of a flexible, autonomous picking solution,
leaving for future works the integration with the vision
system.
Some of the most challenging and common situations an
autonomous Pick and Place system might encounter are: 1)
reduced collision-free end-effector poses due to other goods
or containers; 2) restricted portion of the external surface of
the object available for gripper contact due to other goods
especially when tightly packed together; 3) deformability of
the object, meaning that the shape of the grasped objects
changes under external forces; 4) porosity of the object,
which prevents the employment of simple and nimble suction
grippers. Despite the great effort in the development of
picking solutions, as far as the authors know, no existing
automatic solution is flexible enough to cope with such
challenges, many of which may occur together. A detailed
problem statement is reported in Sec. III, while an overview
of the existing solutions on the market is reported in Sec. II.
The main contributions of this work are the design, realiza-
tion, and testing of WRAPP-up, a novel human-inspired dual
arm robot for intralogistics. The development of WRAPP-
up relies upon observing the techniques adopted by human
pickers at work in warehouses. Indeed, by observing expert
operators, we identified four main maneuvers they commonly
adopt, which are detailed in Sec. IV. Based on these findings,
we designed a dual arm robot (see Sec. Vfor more details)
composed of two 7 degrees-of-freedom manipulators and two
different end-effectors: an adaptive end-effector able both
to grasp a large variety of objects and to stably interact
with different shapes, and a tray with an actuated belt.
Moreover, we encoded the human observed picking strategies
into parametric motion primitives adopted in the trajectory
planning of the robot. Finally, an extensive experimental
validation (see Sec. VII) has been conducted.
To the best of the authors’ knowledge, WRAPP-up is
the first autonomous picking system able to approach the
whole spectrum of picking tasks of the intralogistics: from
bin picking to pallet picking.
II. REL ATE D WORK
Picking tasks can be classified based on different parame-
ters, of which one of the most important is the location from
which the items should be grasped.
On one side, there is the bin-picking problem in which the
objects are typically placed either in an ordered way (or not)
in a box and a single object has to be grasped. Often the
object has size and weight such that it can be handled with
one end-effector. The problem of grasping a single object
with an ad-hoc end-effector has been extensively studied
from theoretical and experimental viewpoints. However, bin-
picking still represents an open challenge, especially in un-
structured environments. This is also testified by challenges
aimed to enhance the warehouse automation in picking
operations, such as the Amazon Picking Challenge [12]. The
interested reader is referred to [13] for a comprehensive
review of robotic picking and to [14] and [15] for recent
results.
On the other side, there is the picking of items that are
located on a pallet. To automate this task two main different
solutions can be adopted: a mobile manipulation approach
in which the robot is provided with a mobile base, or a
grounded manipulation approach in which the pallet (or the
shelf) is brought to the manipulator by mobile devices [16].
Prominent examples of autonomous mobile manipulation
platforms for logistics include: Little Helper III [17], DLR
omniRob [18], and Handle [19]. The first two robots are
mainly devoted to picking objects from shelves. They consist
of a robot arm with a two-fingered parallel gripper mounted
on a stable mobile base. Handle instead has an unstable
two-wheeled mobile base, which requires a more expensive
control but substantially reduces the robot’s footprint, is
equipped with a vacuum gripper, and is devoted to box
handling. The interested reader is referred to [20] for com-
prehensive literature reviews on the subject. Furthermore,
Magazino [21] and InVia Robotics [22] sell two products
on the market, mainly devoted to box picking and based on
suction cups. Both the solutions exploit a picking strategy
based on box sliding, which may not be suitable for boxes
stacked one upon the other and, in general, not free to slide.
TORU, the robot by Magazino, is suitable for picking small
boxes from shelves, especially shoe boxes. It has also been
integrated with a different picking strategy, always based on
objects sliding, and additionally requiring the accessibility of
the rear surface of the box [23].
A recent example of a grounded manipulation approach
is the Dora Picker [24]. Its novelty relies on the soft and
adaptive design of the end-effector. The ground-based depal-
letizing robots available in the market, despite their different
working principles, share the drawback of being bulky and
not easily relocatable [25]–[27]. Moreover, unlike WRAPP-
up, they are usually suitable solely for pallet picking, while
a different robot would be necessary for bin-picking. See,
e.g., the example of Swisslog [28], which proposes on
its website two robotic solutions, one for picking larger
boxes, ACPaQ, and one for bin-picking, ItemPiQ. Another
key distinguishing aspect of WRAPP-up compared to the
integrated solutions proposed by Swisslog is that the former
is aimed to incorporate in a unique platform both picking
and discharging.
Currently, the most flexible autonomous systems that may
be used to accomplish picking tasks are provided with
mechanical and vacuum end-effectors. A wide overview of
the most recent development in gripping devices can be
found in [29].
Mechanical end-effectors for prehensile tasks are certainly
the most widespread and many of them fall into two neatly
distinct categories: simple grippers [30], [31] and complex
or anthropomorphic hands [32]–[34].
Among them, several devices exist that trade simplicity for
flexibility. Examples include underactuated and soft grippers
[35]–[37] and end-effectors with active surfaces. An example
of a versatile mechanical gripper is the Traction Gripper
[38]. It has a shaped frame with counter-rotating belts, which
exploits the friction forces to pull the boxes towards the
corners of the frame and firmly hold them in position. An
evolution of this concept is the Fraunhofer Roll-on Gripper
[39], which is a hybrid between a lift and a gripper. In
this solution, the belts also allow for manipulating (translate
or rotate) the boxes once picked up by the gripper. A
similar solution is exploited by Premium Robotics [25].
These grippers alone show limitations when the grasped item
is lodged in the object below.
Vacuum grippers are widely employed for the grasping
of boxes by their top surface ( [26], [27]), e.g., [40]. The
vacuum gripper by Wynright Robotics [41] is able, as well
as others ( [21], [22]) to grasp from the frontal side. The
device relies on an array of vacuum cups to drag the box on
a support surface of the gripper.
Vacuum grippers show severe drawbacks when the inter-
ested surface of the object is not suitable to be grasped, e.g.,
because the top surface is not present at all, or it is not robust
enough to sustain the weight of the object.
III. PROB LEM DE FINI TIO N
The task of interest, in this work, consists of picking
several different goods from single-item pallets, namely
pallets composed of several units of an item. The input to
the picking system is the sequence of goods to be picked
and their location on the pallet that may be provided by
a vision system. The design of a robot for picking tasks
depends on the size and the shape of the objects that must
be manipulated, and on the modes that can be profitably used
to grasp them. Picking tasks, the vast majority of which is
currently executed by human operators, can be classified into
two main categories: the ones that can be accomplished with
a hand and the ones that must be accomplished with two
hands. The problem of picking objects with a single end-
effector has been extensively treated in the literature and
in previous works of the authors [42] and [15]. This work
is focused on the problem of picking objects that humans
cannot pick with one hand. Notably, the solution that will
be proposed in this work will be able to accomplish both
categories of picking tasks. In the following, a list of items
that represents the 40% of the volume of a food warehouse
is reported together with their main features. Notice that the
food & beverage is one of the market segments most affected
by the e-commerce revolution, which today allows customers
to have their shopping bags directly delivered at home. In the
next subsections, the functional requirements that a picking
system should satisfy to work in such a warehouse profitably
are stated. Finally, the main challenges to be tackled in the
design and realization of this device are described.
A. Objects
The complete list of objects to be picked with their size
and weight is reported in Table II. The objects can be
grouped into two sets, depending on their shape: boxes or
cylinders. For the boxes, the values for the length (L), height
(H), and width (W) expressed in centimeters are reported,
while for the cylinders, the values for the diameter (D) and
the height (H) are listed.
B. Functional Requirements
A list of functional requirements that a picking system
should match is reported in Table I. It is important to stress
that they should not be taken as absolute values for every
intralogistics company but for operators that manage a set
of objects comparable to the one reported in Table II. These
are grouped into key performance areas and indicators. Their
quantitative value should be considered as a target for the
picking system.
TABLE I
PICKING TASK TARGET PERFORMANCE
Performance Area Performance Indicator Target Unit
Productivity Average time to empty a pallet 123 min
Picks per hour 180 #
Reliability First-Attempt Success Rate 90 %
Productivity performance indicators have been computed
based on the fact that such a robot would be economically
sustainable if it is able to perform 3 picking movements every
minute (corresponding to 180 picks per hour). Given that
a Euro-pallet (80 x 120 cm base piled up to 1.5 meters)
may contain up to 627 smallest items, and up to 115 largest
items (among the objects reported in Table II), the time
needed to empty a pallet can be evaluated in 209 and 38
minutes, respectively. The average of these two values gives
the productivity performance indicator reported in the table.
The picking success rate takes into account the grasping
system without considering the vision system.
C. Challenges
The main challenges of the picking phase can be identified
in the following 3 points:
boxes are often very close to each other, with two op-
posite sides, which are the most desirable for a reliable
and robust grasp, usually not easily accessible. Hence,
to be properly handled, they should be first moved to
TABLE II
OBJECTS CONSIDERED FOR PICKING TASKS,WITH THEIR WEIGHT (KG), S IZE (C M), AN D REL ATE D GRASPIN G STRATE GIES
9.1kg - 22×26 4.3kg - 20×25 3.2kg - 16×18 3kg - 20×13
Horizontal Rotation Horizontal Rotation Horizontal Rotation Horizontal Rotation
3kg - 19×29×14 2.7kg - 40×7×30 2kg - 40×8×30 1.8kg - 15×13
Horizontal Rotation Horizontal Rotation Horizontal Rotation Horizontal Rotation
9kg - 46×15×18 6.0kg - 40×10×26 5kg - 46×18×15 2.5kg - 38×18×14
Vertical Rotation - Sliding Vertical Rotation - Sliding Vertical Rotation - Sliding Vertical Rotation - Sliding
2.5kg - 36×15×23 2.0kg - 38×23×20 1.9kg - 35×13×16 0.4kg - 40×18×25
Vertical Rotation - Sliding Vertical Rotation - Sliding Vertical Rotation - Sliding Vertical Rotation - Sliding
(a) Human grasping a box–shaped object.
(b) Human grasping a cylinder–shaped object.
(c) Human grasping a cylinder–shaped object.
(d) Human grasping a box–shaped object.
(e) Human grasping a box–shaped object.
Fig. 2. Human grasping different shaped objects with different strategies.
guarantee that two opposite faces are accessible and then
picked.
some of the items do not have a top surface, or it may
not be suitable to grasp the object. These objects cannot
be grasped with vacuum grippers.
the bottom side of some objects is recessed in the upper
side of the objects which lie under, or more in general,
they can not slide. This means that the objects can only
translate along the vertical direction or rotate about a
horizontal axis.
IV. HUMAN PICKING SKILLS
There is not a systematic method to synthesize a system
that matches all the requirements listed in Sec. III, and one
of the reasons is that it simultaneously involves the co-
design of the robot structure, planning, and control. Hence
we observed skilled human operators at a food warehouse
during the execution of manipulation tasks when picking the
objects listed in Tab. II. More in detail, we recorded two
human operators from a food warehouse while performing
the picking action. Each picking action has been repeated
three times. These live observations and the analysis of the
video recordings led to two lessons learned:
bi-manual manipulation has a crucial role in picking
operations since humans use both hands to manipulate
and handle the objects. In the majority of the tasks, one
hand is used to move the object, and the other hand is
used as a support;
the strategies to pick the items are classifiable in three
main classes.
The strategies, shown in Fig. 2, that human operators adopt
while manipulating goods fall into 3 groups depending on
object shape and form factor:
a) Rotation about Horizontal Axis: In case of thin
boxes, i.e., H>W,H>L, cylindrical objects, or if the
support surface of an object cannot slide, the operators use
one hand to rotate the goods about a horizontal axis and to
put the object on the supporting hand (see Fig. 2(a,b,c) and
Fig. 2(d)).
b) Rotation about Vertical Axis: For thick boxes (H<
W,H<L) with no constraints at the base, the horizontal
rotation is not convenient because of the less favorable lever
arm; thus the operators decide to rotate the boxes about a
vertical axis to have access to the back surface of the object,
as in Fig. 2(d). This strategy can then evolve in two different
picking continuations. In the first one, the box is picked up
by two opposite surfaces while the operator uses his hands
like the jaws of a parallel gripping device. In the second
one, the box is first dragged towards the worker acting on
the back surface and then supported by the other hand as the
box sticks out the pallet or the underneath layer of goods.
This helps apply two different grasping strategies: grasp the
object relying on contacts on two opposite surfaces (front
and back) and slide the object relying on the contact on the
back surface.
c) Sliding: For thick boxes with no constraints at the
base, the operators push or pull the objects until they reach
the support hand at the boundary of the pallet, as in Fig. 2(f).
Picking strategies that human operators adopt highlight
that often, pickers naturally choose different functions for
each hand. One hand is mainly used to accomplish manip-
ulation tasks: pushing an object in the sliding strategy and
adapting to the shapes of the different objects in the other two
strategies. The other hand is often responsible for supporting
the majority of the item weight and may be used to perform
placing operations such as alignment and unloading.
V. WRAPP-UP DESIGN
Inspired by the techniques adopted by warehouse workers,
the envisioned solution is a dual arm system. The system is
composed of two Lightweight Robots arranged to perform
pick and place tasks properly. The mounting bases of the
arms are fixed at an established relative pose, as better
described in the following. Two different end-effectors, pro-
vided with six-axis force/torque sensors, are attached to the
wrists of the robot arms. To perform the dexterous operation,
one arm is featured with a Pisa/IIT SoftHand: a human-
like, adaptive, robust artificial hand the closure movement
of which is easy to control since it is actuated by a single
motor. The second end-effector is the Velvet Tray, which
serves as a support tool. A brief description of the design of
the two end-effectors is provided at the end of this section.
In our preliminary experimental set up each robotic arm
has been mounted on an independent movable gate, which
allows for three DOFs in a plane (two translations and
a rotation), enabling configurable relative locations of the
two arms (see Fig. 3(c)). With this test-bench, we could
easily test different relative positions of the arms in a store-
like environment. The choice of the most suitable relative
configuration of the arms is described in the next subsection.
In a final use case, the dual arm robot can be mounted on a
fixed base as well as on a mobile base, depending on the end-
user requirements. For instance, the end-user may employ a
picker-to-goods strategy, thus needing a mobile base for the
picking station, or a goods-to-picker strategy, in which the
picking station is fixed and the objects are brought there. At
the moment, our robot is mounted on a fixed base, but it
will be integrated on an autonomous mobile robot in future
works.
A. Relative pose of the two manipulators
Once the overall structure of the robot, the end-effectors,
and the manipulation strategies have been defined, the robot
design can be detailed. Particularly important is the location
of the two arms w.r.t. each other and the pallet. A wrong
relative location of the arms may prevent the correct execu-
tion of a strategy because of two main reasons: one joint (or
more) reaches the limit of its range of motion, or a point of
the desired trajectory is out of the reachable workspace of
the bimanual system.
To properly choose the relative location of the arms, a two-
step strategy has been adopted. First, a manipulability index
for each arm has been evaluated for a set of points to find
a relative pose that provides an adequate superposition of
the manipulators’ dexterous workspaces. Then, a feasibility
analysis, conducted by simulating the kinematic execution
of the robot trajectories (based on the strategies presented in
Sec. VI for the objects listed in Tab. II), has been performed
to check that the relative pose found in the first step allowed
the robot to operate at least over half of the pallet footprint.
The feasibility phase took into account constraints due to
joint limits and realistic external obstacles, e.g., floor and
shelves. It is worth pointing out that, for the first step of the
strategy, other metrics could have also been implemented,
e.g., to include the direction of maximum force of the
arms. Still, at this stage, we preferred to give priority to
the manipulability. Future works will be devoted to the
evaluation of different metrics. Among the manipulability
(a) An example of the manipulability analysis showing the manipulability
measure and the preferred directions of motion.
(b) Configuration of the two arms in
the current setup. x=20cm, z=
30cm, θ=15deg (θR=45 deg and
θL=60deg).
(c) CAD Model of the experimental
setup
Fig. 3. Analysis of the relative configuration of the manipulators, resulting relative configuration, and CAD Model of the experimental setup.
TABLE III
MAN IPUL ABI LITY A NALYS IS FOR D IFFE RENT CO NFI GU RATIO NS O F T HE T WO A RM S
Workspace Volume - VU(m3) 5.91 7.13 6.43
Shared Workspace Volume - VI(m3) 2.60 1.38 1.23
Intersection Average Manipulability ¯wD|I0.43 0.46 0.47
Manipulability M0.19 0.09 0.09
measures suitable to quantify the ability of a robot to execute
a movement in an arbitrary direction of the Cartesian space,
from a given pose q, we use the one presented in [43]
w(q) = qdet(J(q)J(q)T),(1)
that is a measure of the volume of a six-dimensional ellipsoid
of which the semi-axes length is represented by the square
roots of the singular values of the end-effector Jacobian J(q).
The eigenvector of J(q)corresponding to the largest singular
value represents the easiest direction of motion. Fig. 3(a)
represents an example of a graphic result that shows the
manipulability index w(q)(evaluated according to Eq. 1) and
the preferred directions of motion of the two robots.
Given a relative location of the arms, we evaluate the
manipulability of the configuration using a scaled average
manipulability for the two arms [44] and the volume of the
shared workspace. The average manipulability of each arm is
evaluated by averaging the manipulability index values at N
uniformly-sampled feasible configurations in the joint space.
The maximum value manipulability index then scales this
value according to
¯wi=N
j=1wi(qj)
maxj{wi(qj)}N.(2)
Then, we defined the manipulability index for the dual arm
(a) The Pisa/IIT SoftHand is the dex-
terous end effector of WRAPP-up.
Furthermore, the picture shows also
all the components attached to the
wrist of the robotic arm.
(b) The Velvet Tray is the support end effector of Wrapp-up. On the left a 3D view is shown. On the right a cut
view shows the power transmission group and the conveyor belt.
Fig. 4. The Manipulation End-Effector (Pisa/IIT SoftHand) and the Support End-Effector (Velvet Tray).
system as
¯wD=¯w1+¯w2
2(3)
where ¯w1and ¯w2are the average manipulability indexes for
the first and second arm, respectively. Thus, the configuration
manipulability is then expressed as
M=VI
VU
¯wD|I(4)
where we defined with VUthe workspace of the dual arm
system (obtained by the union of the workspaces of the two
manipulators), with VIthe volume of the shared workspace,
and with ¯wD|Iwe denote the average manipulability index
for the dual arm system (defined as in (3)) computed using
only the configurations belonging to the intersection of
the workspaces of the two manipulators. The poses with
high manipulability are the ones that give large dexterous
collaborative workspaces.
The solution provided by the manipulability analysis is
a reasonable starting point. However, its selection does not
take into account the tasks that the robot should execute.
To evaluate the quality of the selected configuration for our
task, we simulated the execution of picking tasks using the
strategies described in Sec. IV for different positions of the
target object and registered the associated Cartesian error. To
perform this analysis, we considered the object as they were
placed on a 0.8m×0.6m×1.5m pallet in front of the robotic
platform so that it would lie within its reachable workspace.
The width corresponds to half of the width of a EURO-pallet.
We simulated the task for every poses an object could assume
on this reference pallet (the possible poses are limited and
depend on the shape and the dimension of the object), and
we checked that, in the selected configuration, the robot was
able to execute the task with a limited Cartesian error (under
1cm of error).
To reduce the complexity of the search of the relative
configuration, we predefined a set of reasonable candidate
poses and evaluated the manipulability for each of them and
evaluated the task performance for the most promising one.
Examples of the manipulability for three different candidate
configurations are reported in Tab. III, where the values have
been obtained using N=20000 samples. It is worth noting as
the first configuration (the one we have eventually selected)
is the one with the higher manipulability. A detail of the
manipulators’ base relative location is depicted in Fig. 3(b).
B. Mechanical Design of the Manipulation End-effector
The manipulation end-effector is the Pisa/IIT SoftHand
(Fig. 4(a)). Its mechanical robustness and adaptability, to-
gether with the ease of control, make it particularly suitable
for the type of use required to accomplish the picking task.
For an in-depth description of the hand the reader can refer to
[45]. The Pisa/IIT SoftHand (7) is attached to the wrist flange
of the robotic arm (1) through a 6-axes Force/Torque ATI-
Mini45 sensor (2) and four rubber beams (6). The ATI sensor
detects changes in the state of the hand, such as contact
with the objects to be manipulated in regular functioning but
also undesired collisions, preventing the end-effector from
damaging. The rubber beams are located between the end-
effector and the ATI-sensor. They favor the slowdown of the
external force loading rate in case of collision, increasing the
time for a rapid emergency-stop response. Toothed flange (5)
crimps plate (3) (fixed to the sensor) and plate (4) (fixed to
the hand side) together.
C. Mechanical Design of the Support End-effector
The end-effector that functions as a support for the goods
to be manipulated is the Velvet Tray. Fig. 4(b) shows a
3-D model of the Velvet Tray. Its design and principle of
operation are inspired by research carried out by the Authors
about grippers with active surfaces [46]–[48]. It is equipped
with an actuated belt to ease the loading maneuvers of
the goods. It is attached to flange wrist (1) of the KUKA
arm through flange (2). Between the KUKA arm and the
Velvet Tray, a 6-axes Force/Torque ATI-Mini58 sensor (3)
and rubber beams (4) are interposed with the same aim as in
(a) Strategy to grasp an object from behind (side view). (b) Strategy to roll a tall object (side view).
(c) Strategy to rotate an object exploiting environmental constraints, such as
another object used as a pivot point (top view).
(d) Strategy to lift an object and put it on the tray (side view).
Fig. 5. Robot grasping strategies.
the Pisa/IIT SoftHand. The elastic junction is here composed
of 10 rubber beams (4) arranged in a circle, which slow down
the loading rate of the external forces in collision events. Belt
(8) is coated with high grip polyurethane with a coefficient
of static friction polyurethane-steel of 0.8 which allows an
inclination of 38with respect to a horizontal plane without
a mass on the belt sliding down. This value of the friction
coefficient provides the worst-case for the torque of the motor
that guarantees to hold the target mass of 2.5kg with the
Velvet inclined 38. A Maxon motor DCX22 actuates the
belt with gear-head GPX83 (5) that is able to move a mass
of 2.5kg with an inclination of 38within the continuous
functioning condition of the driver. The power transmission
between the motor and the driver roll of the belt is due to
gears (6). Tension roller (7) ensures a proper tension in the
belt of at least 20Nnecessary to transmit the required torque.
Finally, a set of idle rollers (9) sustain the objects and form
an approximately flat surface under the belt.
VI. TR A NS LATING HUMAN PICKING SK IL LS IN TO
ROB OT MOT ION PRIMITIVES
Inspired by the observation of the strategies adopted by the
operators (described in Sec. IV) parametric motion primitives
have been defined to plan the motion of the robot during the
task execution:
Sliding one end-effector is used to push (or pull) an
object towards the other end-effector, which secures the
grasping and may support the weight of the object, as
shown in Fig. 5(a).
Horizontal Rotation one end-effector is used to tilt
the object about a horizontal axis gently. This can be
achieved in two different cases: i) about a horizontal
axis on the front-bottom edge of the bounding box
enveloping the object (see Fig. 5(b)); ii) about an axis on
the back-bottom edge of a bounding box enveloping the
object (see Fig. 5(d)). In the first case, the object rotation
will end when it lays on the support end-effector. This
strategy is intended to be used with objects (boxes,
cylinders) of which the height is the largest dimension.
In the second case, the object’s rotation will allow the
second end-effector to be placed under the object as
support.
Vertical Rotation one end-effector approaches the ob-
ject’s side, then rotates it about a vertical axis ideally
located at one edge of the object. Once the rotation
has produced enough room for the end-effector, it slides
inside this gap and proceeds to slide the object towards
the pallet’s exterior. This strategy is suitable when the
objects are compactly packed (see Fig. 5(c)), and it is
necessary to make room for the end-effectors to perform
a successful grasp.
For each object, the choice of the strategy has been made
based on its shape and on how the objects are stacked on the
pallet. Each motion primitive is defined as a set of Cartesian
waypoints for the two end-effectors, expressed w.r.t. a frame
placed on the object. The definition of waypoints that allows
for a correct manipulation of the object, e.g., to rotate or
tilt it in order to produce enough room for positioning an
end-effector as in 5(c) or 5(d), is the result of simulations
and real experiments on the objects. Thus, they depend on the
physical properties of the end-effectors and the objects. Once
the pose of the object is retrieved, and the correct primitive
is selected, the waypoints expressed in the object-fixed frame
are transformed in the world frame.
In order to take into account the robot kinematics and the
joint limits for the motion planning, we determined the path
at the joint level via the reverse priority algorithm described
in [49], which allows us to define a set of tasks with different
priorities including unilateral constraints (e.g., joint-position
limits). More in detail, for each of the two arms, we have set
the Cartesian pose tracking, i.e., the position and orientation
of the end-effector, as the low priority task, and the joint-
(a) Reactive planning architecture. The picking strategy is divided into
basic consecutive phases planned online. The information from the sen-
sors are used to detect contacts between the robot and the environment,
or if the end-effectors reached their target position. Depending on the
typology of the picking phase (Approach or Manipulation), these events
trigger the transition to the next phase or an emergency state.
(b) Force sensors readings for the two end-effectors during the strategy
execution. The transitions between the states into which the strategy is
decomposed are highlighted.
(c) Sequence of a picking strategy in which each block corresponds to a state of the state machine on which the planner is built.
Fig. 6. Reactive Planning Approach.
position constraints as the high priority tasks. Then the path
following could be achieved by the minimum-time approach
presented in [50].
Accounting for realistically imperfect knowledge of the
object position provided by a vision system in a future
integration, the force/torque sensors have been exploited
to plan the trajectories based on contacts with the objects
reactively. Indeed, the measured forces can be used to detect
possible contact with an object whenever they exceed a user-
defined threshold. Our reactive planning approach (see Fig.
6(a) for a schematic representation of the architecture) is
accomplished by decomposing each picking strategy into
consecutive basic phases represented by states of a finite
state machine. Each phase is planned online (block ”Plan
Picking Phase” of Fig. 6(a)) and generates the Cartesian
trajectory for the end-effectors. For each of the phases, the
kinematic feasibility of the planned trajectory is checked
(block ”Feasible” in Fig. 6(a)). In this block, the generation
of a path for the joints of the robot is performed using
the reverse priority algorithm. If the desired motion is not
feasible because, e.g., of the constraints on the joint ranges,
the task is aborted, allowing the intervention of a human
operator. The transition between a phase and the following
is triggered online based on the information coming from
the force/torque sensors and the joint position sensors of the
robot. This information can be used to detect two possible
events (block ”Event” in Fig. 6(a)): a detected contact (or
the loss of contact) or the end-effectors reaching their target
position. The reaction of the system at these events depends
on the typology of the picking phase. Indeed, the phases are
of two types: Approaching phases and Manipulation phases.
Approaching phases are the ones in which the end-effectors
have to establish contact with the object. This is a critical step
of the manipulation process, since an incorrect positioning
of the end-effector w.r.t. the object, possibly due to errors
on the estimate of the object pose, could cause the picking
failure. Hence the end-effector starts moving towards the
object along a specified direction until contact is detected.
Then, it stops, and the end-effector position at the contact is
used to update the object pose estimate. This refined estimate
is used to update the planned trajectory for the following
phases. If the expected contact does not happen within a
certain region, the system enters an emergency state, and
eventually, a human operator will be alerted. Conversely,
Manipulation phases are the ones for which a contact is
already established, and the end-effectors are manipulating
the object. The condition used to trigger the transition to
the successive phase is defined based on the end-effector
reaching the target position. Sensing of unexpected forces,
causes the system to enter an emergency state and eventually
alert a human operator, regardless of the specific phase. At
the end of the picking strategy, since the object will be placed
on the velvet tray, the force measurements can be used to
identify whether the object has been picked or if it fell.
(a) Horizontal Rotation Primitive.
(b) Cylindrical objects being picked using the horizontal rotation strategy.
Fig. 7. WRAPP-up picking cylindrical objects.
An example of the described reactive approach is reported
in Fig. 6(c) and 6(b), where the states of an example trajec-
tory and the corresponding force sensor readings are shown.
For this example, we considered a worst-case scenario where
the uncertainties on the pose estimate are such that require
additional pose refinement steps. Indeed, in phase (A) the
hand is approaching the object laterally to refine its pose. As
reported in Fig. 6(b), when contact is correctly detected (the
magnitude of the force Fxalong the contact direction exceeds
the threshold set at 10N), the pose along this direction is
updated, and the hand is placed front to the object and
starts moving towards it. Once again, the force measurements
inform about the established contact, and the robots enter
the third phase, (C), of the manipulation. In this case, since
a Horizontal Rotation is used, the hand lifts the bucket to
create space for the velvet tray. Therefore, the transition
toward the next phase is triggered by the hand reaching the
target Cartesian pose, and no force information is required.
Note that, to increase the robustness of the system, the force
measurements in this phase could detect the loss of contact
between the hand and the object and be used to abort the
current picking action. Regarding the other phases, contact
information is again used to trigger the transition from (D)
to (E) (the velvet tray is placed under the object, in contact
with it), and (E) to (F), where the hand is approaching the
object from the top to perform a collaborative sliding.
As said, the example shows the effectiveness of the
reactive approach even in case of a not precise knowledge
of the pose, which although requires the execution of the
redundant and time-consuming lateral approach (phase (A)).
Depending on the level of uncertainties on the pose estimated
by the perception system, such redundant steps could be not
necessary.
VII. EXP ER IME NTAL VALIDATION
In this section, the preliminary experimental validation
of WRAPP-up is reported. The picking capabilities of the
system are shown on a representative set of objects from the
ones in Tab. II. Indeed, the objects used for the tests allow us
to cover the two main shapes we have identified to be relevant
for logistics, i.e., cuboids and cylinders. Furthermore, they
allow us to test all the four motion primitives we presented
(a) Horizontal Rotation Primitive.
(b) Two rows (8 pieces) of thin boxes being picked using the horizontal rotation strategy.
Fig. 8. WRAPP-up picking thin boxes.
and validate the platform’s performance in different picking
scenarios.
Fig. 7(a) shows the strategy used to manipulate objects
which have a characteristic cylindrical shape, better suited
for a horizontal rotation strategy. With this approach, the
hand is placed in front of the bucket and grasps its edge
allowing it to lift and tilt it. This movement allows the tray
to be placed beneath it as a support. Once the tray has been
correctly positioned, the hand can release the object, see e.g.
the last frame of Fig. 7(a), and the tray is used to collect and
deploy the bucket. In this case, the hand can be employed
to ease the tray during the picking phase.
An example of the described approach used to collect three
rows of objects placed on a pallet is shown in Fig. 7(b).
The horizontal rotation strategy is also effective to pick
thin boxes, see Fig. 8(b).
In this case, the hand is placed behind the box and tilts it
until the box lays on the tray placed in front of the object
with a proper inclination. Then, the hand is used to ease
the object picking, keeping it on the tray while the latter
is returning parallel to the horizontal plane. The former
approach has been tested to successfully pick 8 boxes close
to each other, as in Fig. 8(b), showing the robustness of
the designed strategy even in the presence of other objects
behind the handled box.
The best picking strategy is not chosen solely based on
the object’s shape, but it also depends on the location of the
object on the pallet and the position of the other possible
items. To show this concept, two different picking tests have
been performed on the same object (with a box-like shape)
depending on its different orientation, see Fig. 9and Fig. VII.
In the first test, the boxes are easily picked using a sliding
approach due to their configuration. The hand is placed
behind the box and used to pull the object towards the
tray. The situation is different and requires a more complex
strategy; if the boxes are in a different configuration, e.g.,
they are rotated by 90 degrees around the vertical axis w.r.t.
the previous case, and they are compactly packed, as shown
in Fig. VII.
This condition requires the use of a vertical rotation
strategy, where the hand approaches the box’s side and, eased
by the tray, rotates it about a vertical axis located at one edge.
Fig. 9. WRAPP-up picking thick boxes using sliding primitive.
Then, the hand slides inside the created gap and proceeds to
slide the box towards the tray.
Table IV shows the time for picking every single object
during the performed experiments. Then, an estimation of the
time to empty an entire pallet full of that object is reported.
To estimate the total number of boxes that are contained
in the pallet, the standard EU Pallet dimensions have been
considered for the base of the pallet, and a full pallet has
been considered to be 1.5 meters height. To compute the
number of objects that can be contained in such a pallet,
the dimensions of the objects have been taken into account.
A pallet of thin boxes contains thus 192 items, a pallet of
cylinders 360 items, one of the thick boxes 176 items in the
first case, and 165 items in the other. Hence, the time to
empty a pallet has been estimated, multiplying the average
time to pick an object for the number of objects in the full
pallet. Table Vreports the global performance indicators we
obtained for the picking task. The time to empty a pallet has
been computed as the average of the values reported in Table
IV. Fifty picking actions have been performed for each case
in order to test the system and estimate the values of the
performance indicators. The success rate is the average of
the four cases.
A. Discussion
This set of experiments was aimed at verifying the ef-
fectiveness of the hardware and the picking strategies in an
unloading simulation of goods on a pallet. These experiments
demonstrated that the WRAPP-up robotic platform is suitable
to fulfill the picking tasks of goods stacked on a pallet. A
comparison with the performance requirements specified in
Tab. I, highlights that the reliability requirement is met, but
the productivity should be improved through optimization
techniques (not yet integrated) that will be the subject
of future work. The performance reported in Tab. Vare
expressed for the platform in its current setup, i.e., without
a perception system and the optimization module for the
two arms and for the set of objects we used for the tests.
Indeed, the reliability has to be intended as an upper bound
for the system, since it does not include the presence of a
perception system. For the average time to empty a pallet, it
will have to be evaluated for the fully integrated platform, to
quantify the impact of the time required by the perception
system to retrieve the pose of the objects, and the impact
the optimization module presented in [50] in the overall
performance. It is worth noting as, with the current setup,
some of the more burdensome objects in Tab. II could result
in being difficult to handle because they exceed the nominal
payload of the arms. Strategies and solutions to effectively
handle those objects will be investigated. A more specific
study on the robustness of the platform for different objects
and setups, to evaluate how different friction forces could
impact the reliability of our manipulations that involve the
exploitation of environmental constraints, will be subjects for
future works.
VIII. CONCLUSIONS AND FUTURE WOR KS
In this work, we addressed the problem of realizing a
proof-of-concept robot that is flexible enough to manipulate a
variety of goods relevant to the intralogistics of warehouses.
Inspired by the picking strategies that skilled human oper-
ators adopt in the execution of these tasks, we realized a
Fig. 10. WRAPP-up picking thick boxes using the vertical rotation primitive.
TABLE IV
PICKING PERFORMANCE INDICATORS FOR THE FOUR SCENARIOS.
Object
Picking time per
object
55s 83s 16s 82.5s
Time to empty a
pallet
176min 498min 47min 227min
TABLE V
PICKING PERFORMANCE INDICATORS
Performance Area Performance Indicator Current Unit
Productivity Average time to empty a pallet 237 min
Reliability Picking success 92.5 %
dual arm robot provided with a Pisa/IIT SoftHand and a
Velvet Tray. The first end-effector is adaptable; hence it is
used to establish stable grasps to rotate and slide goods with
various shapes, while the second end-effector is mainly used
to support the weight of the objects. The robot has been
experimentally validated in multiple picking actions on a set
of four different representative objects. Future works will
include, on one side, to provide the robot planning with a
high-level decision tool that is able to automatically generate
the right strategy to adopt on the basis of features of the
objects that can be detected by a vision system. On the other,
to adopt suitable feedback strategies based on vision, force
feedback, and tactile feedback in order to improve robot
reliability. Furthermore, the average picking time will be
minimized by adopting suitable optimization algorithms, and
the robot will be provided with a mobile base.
ACKNOWLEDGMENT
This work was supported in part by the European Union’s
Horizon 2020 research and innovation program as part of the
projects ILIAD (Grant no. 732737), and in part by the Italian
Ministry of Education and Research in the framework of the
CrossLab project (Departments of Excellence).
REFERENCES
[1] C. McNair, “Worldwide retail and ecommerce sales: emarketer’s
updated forecast and new mcommerce estimates for 2016–2021,
Industry Report, eMarketing, 2018.
[2] N. Boysen et al., “Warehousing in the e-commerce era: A survey,
European Journal of Operational Research, vol. 277, no. 2, pp. 396–
411, 2019.
[3] United States Census Bureau, “2015-2016 SUSB Employment
Change Data Tables,” [Online] Available: https://www2.census.
gov/programs-surveys/susb/tables/2016/us 4digitnaics emplchange
2015-2016.xlsx?#, [Accessed: Jul. 1, 2020], 2019.
[4] Data USA, “Warehousing and storage [website],” Avaliable at: https://
datausa.io/profile/naics/493/#tmap top occs, [Accessed: Jul. 1, 2020],
2017.
[5] K. J. Roodbergen et al., “Designing the layout structure of manual
order picking areas in warehouses,” IIE Transactions, vol. 40, no. 11,
pp. 1032–1045, 2008.
[6] J. Bloodworth, Hired Six Months Undercover in Low-Wage Britain.
Atlantic Books London, 2018.
[7] K. C. Laudon and C. Guercio Traver, E-commerce: business, technol-
ogy, society. Pearson, 2017.
[8] C. G. Petersen and G. Aase, “A comparison of picking, storage, and
routing policies in manual order picking,” International Journal of
Production Economics, vol. 92, no. 1, pp. 11–19, 2004.
[9] R. Krug et al., “The next step in robot commissioning: Autonomous
picking and palletizing,” IEEE Robotics and Automation Letters,
vol. 1, no. 1, pp. 546–553, 2016.
[10] J. Rudd, A Practical Guide to Logistics: An Introduction to Transport,
Warehousing, Trade and Distribution. Kogan Page Publishers, 2019.
[11] W. Echelmeyer et al., “Performance indicators for robotics systems in
logistics applications,” in IROS Workshop on Metrics and Methodolo-
gies for Autonomous Robot Teams in Logistics, 2011, p. 55.
[12] N. Correll et al., “Analysis and observations from the first amazon
picking challenge,” IEEE Transactions on Automation Science and
Engineering, vol. 15, no. 1, pp. 172–188, 2016.
[13] J. Bohg et al., “Data-driven grasp synthesis - a survey,” IEEE Trans-
actions on Robotics, vol. 30, no. 2, pp. 289–309, 2013.
[14] J. Mahler et al., “Dex-net 2.0: Deep learning to plan robust grasps with
synthetic point clouds and analytic grasp metrics,” Robotics: Science
and Systems (RSS), 2017.
[15] C. Gabellieri et al., “Grasp It Like a Pro: Grasp of Unknown Objects
With Robotic Hands Based on Skilled Human Expertise,” IEEE
Robotics and Automation Letters, vol. 5, no. 2, pp. 2808–2815, 2020.
[16] Grey Orange, “Ranger robot series,” [Online] Available: https://www.
greyorange.com, [Accessed: Jul. 1, 2020].
[17] O. Madsen et al., “Integration of mobile manipulators in an industrial
production,” Industrial Robot: An International Journal, vol. 42, no. 1,
pp. 11–18, 2015.
[18] A. D¨
omel et al., “Toward fully autonomous mobile manipulation for
industrial environments,International Journal of Advanced Robotic
Systems, vol. 14, no. 4, p. 1729881417718588, 2017.
[19] Boston Dynamics, “Handle,” [Online] Available: https://www.
bostondynamics.com/handle, [Accessed: Jul. 1, 2020].
[20] M. Hvilshøj et al., “Autonomous industrial mobile manipulation
(AIMM): past, present and future,” Industrial Robot: An International
Journal, vol. 39, no. 2, pp. 120–135, 2012.
[21] Magazino, “Pick-by-Robot: warehouse automation with Magazino
at FIEGE Logistik,” [Online] Available: https://www.youtube.com/
watch?v=kj8NaHAoLJwfiege%20lo, [Accessed: Jul. 1, 2020], 2017.
[22] InVia, “Introducing inVia Robotics,” [Online] Available: https://www.
youtube.com/watch?v=sCE0a8625tE, [Accessed: Jul. 1, 2020], 2018.
[23] Magazino, “Advanced Robotics - mobile picking robots for ware-
houses by Magazino,” [Online] Available: https://www.youtube.com/
watch?v=1j-RhPENu88, [Accessed: Jul. 1, 2020], 2017.
[24] H. Zhang et al., “Dorapicker: An autonomous picking system for gen-
eral objects,” in 2016 IEEE International Conference on Automation
Science and Engineering (CASE). IEEE, 2016, pp. 721–726.
[25] Premium Robotics, “Robots for logistics tasks,” [Online] Available:
http://premium-robotics.com/index.php?id=6&L=1, [Accessed: Jul. 1,
2020].
[26] ABB, “ABB depalletizing solution,” [Online] Available: https:
//www.youtube.com/watch?time$ $continue=1&v=8MbC9IaNvJM&
feature=emb logo, [Accessed: Jul. 1, 2020], 2019.
[27] FANUC, “Robotic system uses three FANUC palletizing robots to
service nine production lines - sen-pack,” [Online] Available: https:
//www.youtube.com/watch?v= lH2snLVO7w, [Accessed: Jul. 1,2020],
2017.
[28] Swisslog, “Robot-based picking & palletizing for higher warehouse
performance,” [Online] Available: https://www.swisslog.com/en-us/
products-systems- solutions/picking-palletizing- order-fulfillment/
robot-based- robotics-fully- automated, [Accessed: Jul. 1, 2020].
[29] G. J. Monkman, S. Hesse, R. Steinmann, and H. Schunk, Robot
grippers. John Wiley & Sons, 2007.
[30] Schunk, “Schunk grippers,” [Online] Available: https://schunk.com/
de en/gripping-systems/category/gripping-systems/schunk-grippers/,
[Accessed: Jul. 1, 2020].
[31] Willow Garage, “2G ”Velo” Gripper,” [Online] Available: http://www.
willowgarage.com/velo2g, [Accessed: Jul. 1, 2020].
[32] R. O. Ambrose et al., “Robonaut: Nasa’s space humanoid,IEEE
Intelligent Systems, vol. 15, no. 4, pp. 57–63, 2000.
[33] L. Birglen et al.,Underactuated robotic hands. Springer Verlag,
2008.
[34] M. Grebenstein et al., “The DLR hand arm system,” in IEEE Interna-
tional Conference on Robotics and Automation (ICRA). IEEE, 2011,
pp. 3175–3182.
[35] A. Bicchi and A. Marigo, “Dexterous grippers: Putting nonholonomy
to work for fine manipulation,” The International Journal of Robotics
Research, vol. 21, no. 5-6, pp. 427–442, 2002.
[36] S. Hirose and Y. Umetani, “The development of soft gripper for the
versatile robot hand,” Mechanism and machine theory, vol. 13, no. 3,
pp. 351–359, 1978.
[37] A. M. Dollar and R. D. Howe, “Designing robust robotic graspers for
unstructured environments,” in Workshop on Manipulation for Human
Environments, 2006.
[38] Fraunhofer, “Traction gripper,” [Online] Available: https:
//www.iml.fraunhofer.de/en/fields of activity/material-flow-systems/
machines and facilities/developments/traction gripper.html,
[Accessed: Jul. 1, 2020].
[39] Fraunhofer, “Flexible single-piece roll-on-gripper for intralogistics,
[Online] Available: https://www.ipa.fraunhofer.de/content/dam/ipa/de/
documents/Projekte/Forschungsprojekte/300 371 d e.pdf, [Accessed:
Jul. 1, 2020].
[40] J. S. Andersen and T. Christensen, “Apparatus for handling layers of
palletized goods,” October 12 2004, uS Patent 6,802,688.
[41] Wynright Corporation, “Robotic solutions,” [Online] Available: https:
//robotics.wynright.com, [Accessed: Jul. 1, 2020].
[42] M. Bonilla et al., “Grasping with soft hands,” in IEEE-RAS Interna-
tional Conference on Humanoid Robots (Humanoids). IEEE, 2014,
pp. 581–587.
[43] T. Yoshikawa, “Manipulability of robotic mechanisms,The interna-
tional journal of Robotics Research, vol. 4, no. 2, pp. 3–9, 1985.
[44] T. Yoshikawa, Foundations of robotics: analysis and control. MIT
press, 1990.
[45] C. Della Santina et al., “The quest for natural machine motion:
An open platform to fast-prototyping articulated soft robots,” IEEE
Robotics & Automation Magazine, 2017.
[46] V. Tincani et al., “Velvet fingers: A dexterous gripper with active
surfaces,” in 2012 IEEE/RSJ International Conference on Intelligent
Robots and Systems. IEEE, 2012, pp. 1257–1263.
[47] V. Tincani et al., “Implementation and control of the velvet fingers:
a dexterous gripper with active surfaces,” in 2013 IEEE International
Conference on Robotics and Automation. IEEE, 2013, pp. 2744–2750.
[48] R. Krug et al., “Velvet fingers: Grasp planning and execution for an
underactuated gripper with active surfaces,” in 2014 IEEE Interna-
tional Conference on Robotics and Automation (ICRA). IEEE, 2014,
pp. 3669–3675.
[49] F. Flacco and A. De Luca, “Unilateral constraints in the reverse pri-
ority redundancy resolution method,” in 2015 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS). IEEE, 2015,
pp. 2564–2571.
[50] A. Palleschi et al., “Time-optimal path tracking for jerk controlled
robots,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp.
3932–3939, 2019.
... G RASPING objects is a fundamental skill that humans acquire easily, allowing them to manipulate a wide range of objects effortlessly. However, current robotic manipulation and grasping capabilities still lag behind human abilities [1], hindering the development of general-purpose robots capable of operating in unstructured and dynamic environments [2], [3]. Because of this, much effort has been, and is currently being, devoted by the robotics community in studying the theory of grasping with a focus on grasp planning for unknown objects [4]. ...
... The final output of the algorithm is a set B of bounding boxes. Each element b ∈ B is a pair formally defined as 3 ] ∈ R 3 represents the vector of the box dimensions, while ...
... This system allows for the registration of the correct hand pose when grasping a set of 56 cuboid sample boxes, whose dimensions have been chosen to cover the feasible grasp range of the gripper. 3 We also recorded the interaction wrenches exerted during the grasp. Indeed, during the demonstrations, the sample boxes were rigidly fixed and aligned to a sensorized platform. ...
Article
Full-text available
With the improvements in their computational and physical intelligence, robots are now capable of operating in real-world environments. However, manipulation and grasping capabilities are still areas that require significant improvements. To address this, we introduce a new data-driven grasp planning algorithm called Grasp it Like a Pro 2.0. This algorithm utilizes a small number of human demonstrations to teach a robot how to grasp arbitrary objects. By decomposing objects into basic shapes, our algorithm generates candidate grasps that can generalize to different object's geometry. The algorithm selects the grasp to execute based on a selection policy that maximizes a novel grasp quality metric introduced in this article. This metric considers the complex interdependencies between the predicted grasp, the local approximation produced by the basic shape decomposition, and the gripper used. We evaluate our approach against multiple baselines using different grippers and objects. The results demonstrate the effectiveness of our method in generating and selecting high-quality and reliable grasps. With a soft underactuated robotic hand, our algorithm achieves a 94.0% success rate in 150 grasps across 30 different objects. Similarly, with a rigid gripper, it achieves an 85.0% success rate in 80 grasps across 16 different objects.
... M ODERN society is faced with the lack of workforce in various repetitive jobs, such as reshelving products in supermarkets or handling heavy luggage in airports. Robots appear to be the most promising solution to mitigate the negative effects of the declining workforce and perform these various complex tasks [1]. To work in variable and unstructured environments, robots must be dexterous and intelligent to quickly learn the job while interacting safely with other robots, objects, and humans. ...
... While tasks that require only one arm have been explored extensively in the literature, more complex tasks, which require a bimanual setup have only recently been targeted. Among such tasks, picking large objects in unstructured environments ( Fig. 1) [1], assisting the elderly [3], [4], surgery tasks [5], or complex assembly tasks [6] are shown to require dexterous bimanual setups. Factory assembly, logistics, and household applications of bimanual robots have been known for decades [7], [8]. ...
Article
Full-text available
Performing bimanual tasks with dual robotic setups can drastically increase the impact on industrial and daily life applications. However, performing a bimanual task brings many challenges, such as synchronization and coordination of the single-arm policies. This article proposes the safe, interactive movement primitives learning (SIMPLe) algorithm, to teach and correct single or dual arm impedance policies directly from human kinesthetic demonstrations. Moreover, it proposes a novel graph encoding of the policy based on Gaussian process regression where the single-arm motion is guaranteed to converge close to the trajectory and then toward the demonstrated goal. Regulation of the robot stiffness according to the epistemic uncertainty of the policy allows for easily reshaping the motion with human feedback and/or adapting to external perturbations. We tested the SIMPLe algorithm on a real dual-arm setup where the teacher gave separate single-arm demonstrations and then successfully synchronized them only using kinesthetic feedback or where the original bimanual demonstration was locally reshaped to pick a box at a different height.
... Due to the massive growth of e-commerce in recent years, warehouses are focusing on autonomizing repetitive and non-value-added tasks to service the everincreasing package volume flowing into their facilities (Bayhan et al. 2020). The main reason for this is the lack of human capability to perform repetitive and exhaustive tasks at a consistent pace (Garabini et al. 2020). Smart intralogistics systems can perform decentralized decision-making without requiring immediate human supervision (Fragapane et al. 2020). ...
... Smart intralogistics systems can perform decentralized decision-making without requiring immediate human supervision (Fragapane et al. 2020). They can autonomously determine their actions using a perception system and can change or adapt to new requirements (Garabini et al. 2020;Winkler and Zinsmeister 2019). In addition, smart intralogistics systems can also reduce the need for human administration for maintenance and repair. ...
Article
Full-text available
Eyeing superior operational performance, transportation, and warehousing management, researchers have turned their much-needed attention to autonomous and IoT-driven intralogistics systems. Despite its potential, a systematic evaluation of its overall business needs and the criteria for its success at each stage of adoption is missing in the literature. Using the business analysis framework, augmented by the technology adoption model, this study seeks to provide the business context for adopting autonomous and IoT-driven intralogistics by identifying business requirements and critical success factors for such systems. We thematically analyze 85 recent research articles on autonomous and IoT-driven intralogistics systems to identify business requirements that are linked to the mission of maximizing operational profit for the warehousing and storage industry. Then, using the identified business requirements as a base, we thematically analyze those 85 research articles again to identify critical success factors at different stages of technology adoption, namely information, analysis, acquisition, and utilization. We use the findings to develop propositions for future researchers. These findings provide a foundation for developing empirical, descriptive, and normative research on adopting and managing these systems for the warehousing and storage industry.
... obtained within the European projects, such as ILIAD and REFILL projects, which respectively propose autonomous depalletizing for alimentary goods with a fixed bimanual robotic system [9], and solution for single arm object manipulation within shelves of a supermarket [5]. ...
... The manipulation skill is the one that presents the most challenges, related to planning and environment interaction. Fig. 2 shows the structure of our framework for constrained manipulation for the plant logistics application, which build on the research and approaches developed within [9]. To achieve an efficient manipulation capability for the task in consideration we defined grasping and handling strategies inspired by the human behavior. ...
Conference Paper
Full-text available
The machine tending in a productive plant typically requires the transport of material from a storage area to a productive area. The plant logistics phase is a part of the production process that is often performed manually, due to the technological challenges related to the manipulation of objects in constrained environments, such as the shelf of a warehouse. However, an effort for its automation is justified by the fact that for an human operator this activity is fatiguing, not ergonomic and with low added value. This paper proposes a control framework for the automation of plant logistics for an industrial case study, integrating navigation and objects manipulation.
... Planning Domain: In this scenario, we have four agents: an automated forklift, called t; two pallets, called p1 and p2; a fixed base dual-arm robotic platform, WRAPP-up [18], designed to perform autonomous picking and palletizing operations, called r. We have a single object, O = {o}, and three sectors, i.e., S = {σ 1 , σ 2 , σ 3 }, as shown in Fig.1a. ...
... Then, the forklift moves the pallet to the robot station, where the robot picks the object. To plan at the lower level the picking action, we use the reactive planner described in [18] and [19], which allows robust and efficient picking and placing operations for cuboids and cylinders. The average times needed to plan an action for the two agents (t and r) are reported in Tab. ...
Article
Full-text available
Multi-robot systems are becoming increasingly popular in warehouses and factories since they potentially enable the development of more versatile and robust systems than single robots. Multiple robots allow performing complex tasks with greater efficiency. However, this leads to increased complexity in planning and dispatching actions to robots. In this paper, we tackle such complexity using a hierarchical planning framework: the task is first planned at an abstract level and then refined by local motion planning. We propose a framework based on a state-transition system formalism that abstracts the problem by removing unnecessary details and, hence, considerably reduces planning space complexity. Forward search from an initial state allows the robot to find a sequence of actions to accomplish the assigned task. These actions can be planned at a lower level employing any motion planning technique available in the literature. The proposed method is validated through experiments in several operating conditions and scenarios.
... Modern society is faced with the lack of workforce in performing various repetitive jobs like re-shelving products in supermarkets or handling heavy luggage in airports. Robots appear to be the most promising solution to mitigate the negative effects of the declining workforce and perform these various complex tasks [1]. To do so in variable and unstructured environments, the robots must be dexterous and intelligent to learn the job quickly and safely interact with other robots, objects, and humans. ...
Preprint
Full-text available
Performing bimanual tasks with dual robotic setups can drastically increase the impact on industrial and daily life applications. However, performing a bimanual task brings many challenges, like synchronization and coordination of the single-arm policies. This article proposes the Safe, Interactive Movement Primitives Learning (SIMPLe) algorithm, to teach and correct single or dual arm impedance policies directly from human kinesthetic demonstrations. Moreover, it proposes a novel graph encoding of the policy based on Gaussian Process Regression (GPR) where the single-arm motion is guaranteed to converge close to the trajectory and then towards the demonstrated goal. A modulation of the robot stiffness according to the epistemic uncertainty of the policy allows for easily reshaping the motion with human feedback and/or adapting to external perturbations. We tested the SIMPLe algorithm on a real dual arm set up where the teacher gave separate single-arm demonstrations and then successfully synchronized them only using kinesthetic feedback or where the original bimanual demonstration was locally reshaped to pick a box at a different height.
... In [27], a reactive trajectory planner is proposed and tested on a bin-picking use casein which deviations of the generated trajectory are triggered by the proximity with the environment, sensed thanks to an artificial skin on the robot. In [28] and [29], a reactive planner for bi-manual picking tasks in logistics is introduced. Contact detection is exploited to adjust pre-defined motion primitives on the fly so as to adapt to a non-perfectly-known environment. ...
Article
Full-text available
In recent years, robotics has been largely applied to improve the efficiency of logistic processes. Pallets cover a crucial role in the logistic flow, since they represent the main way to store and ship items. When put onto pallets, the items are wrapped with plastic films to protect them and prevent them from falling. Despite being the first and necessary operation for handling the stacked goods, unwrapping-the task of removing the plastic films wrapped around the goods-has not yet been satisfactorily automated. We propose the first robotic solution for autonomous unwrapping of generally shaped pallets, including both homogeneous and heterogeneous pallets. Force and torque measurements are exploited to retrieve information on the collisions between the end-effector and the wrapped items or the plastic film. Based on the contact information, we design a novel reactive planning strategy that makes the unwrapping task effective and robust on pallets with uncertain position or shape. We present the results of an extensive experimental campaign to validate the proposed method.
Article
In this article, we develop a novel tricriteria coordinated motion planning scheme for the redundant dual-arm robot at the velocity level. The optimization criteria are designed to avoid problems for both joint angle drift and kinematic singularity of the redundant robotic arm. First, we establish the preliminary optimization equation by putting together criteria including minimum velocity norm, repetitive motion planning, and maximum manipulability. Meanwhile, the physical limitations of each joint are viewed as constraints. Second, this preliminary optimization equation is transformed into a constrained convex quadratic programming problem so that various solvers can solve the optimization equation and guarantee the optimality of the solution. Finally, we generalize the optimization strategy to dual-arm robots by solving the optimization equations of the two robotic arms in a unified framework. Experiments on a virtual redundant dual-arm robot are carried out to validate the effectiveness and feasibility of the proposed coordinated motion planning scheme.
Article
This paper proposes a combined planning and optimization method that enables a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot’s dual-arm motion that pulls the rope of the crane pulley blocks to lift up the plate. The crane pulley blocks reduce the payload that each robotic arm needs to bear. When the plate is lifted up to a satisfying pose, the planner plans a sliding-pushing motion for one of the robot arms to tumble over the plate while considering force and moment constraints. The article presents the technical details of the planner and several experiments and analysis carried out using a dual-arm robot made by two Universal Robots UR3 arms. The influence of various parameters and optimization goals are investigated and compared in depth. The results show that the proposed planner is flexible and efficient. This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets. Note to Practitioners—This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets.
Article
Full-text available
This work proposes a method to grasp unknown objects with robotic hands based on demonstrations by a skilled human operator. Not only are humans efficacious at grasping with their own hands but are also capable of grasping objects using robotic hands. Therefore, we consider how the grasping skills of a human trained in robotic hand use can be transferred to a robot equipped with the same hand. We propose that a skilled human user manually operates the robotic hand to grasp a number of elementary objects, consisting of different boxes. A Decision Tree Regressor is trained on the data acquired from the human operator to generate hand poses able to grasp a general box. This is extended to grasp objects of unknown and general shape leveraging upon the state-of-theart Minimum Volume Bounding Box decomposition algorithm that approximates with a number of boxes the point cloud of the object. We report on extensive tests of the proposed approach on a Panda manipulator equipped with a Pisa/IIT SoftHand, achieving a success rate of 86.7% over 105 grasps of 21 different objects.
Article
Full-text available
This paper presents a new approach to solve the Time-Optimal Path Tracking under limited joint range and bounds on velocity, acceleration and jerk. To obtain smooth and continuous accelerations, with beneficial effects for the load and wear on the actuators but a limited impact on performance, we state the minimum-time path tracking problem with the jerk as the control input. The main contribution of this paper is a formulation that includes the jerk constraints in the optimization problem and that, even if the resulting Non-Linear Programming (NLP) problem is non-convex, allows to perform an efficient and reliable convex relaxation using McCormick Envelopes. Simulations and experimental tests on a real 7-DoF manipulator have been carried out to show the benefits of the proposed approach and to compare it to state-of-the-art techniques.
Article
Full-text available
This work presents a concept for autonomous mobile manipulation in industrial environments. Utilizing autonomy enables an unskilled human worker to easily configure a complex robotics system in a setup phase before carrying out fetch and carry operations in the execution phase. In order to perform the given tasks in real industrial production sites, we propose a robotic system consisting of a mobile platform, a torque-controlled manipulator, and an additional sensor head. Multiple sensors are attached which allow for perception of the environment and the objects to be manipulated. This is essential for coping with uncertainties in real-world application. In order to provide an easy-to-use and flexible system, we present a modular software concept which is handled and organized by a hierarchical flow control depending on the given task and environmental requirements. The presented concept for autonomous mobile manipulation is implemented exemplary for industrial manipulation tasks and proven by real-world application in a water pump production site. Furthermore, the concept has also been applied to other robotic systems and other domains for planetary exploration with a rover.
Article
Full-text available
To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset of 6.7 million point clouds, grasps, and robust analytic grasp metrics generated from thousands of 3D models from Dex-Net 1.0 in randomized poses on a table. We use the resulting dataset, Dex-Net 2.0, to train a Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly classifies grasps as robust from depth images and the position, angle, and height of the gripper above a table. Experiments with over 1,000 trials on an ABB YuMi comparing grasp planning methods on singulated objects suggest that a GQ-CNN trained with only synthetic data from Dex-Net 2.0 can be used to plan grasps in 0.8s with a success rate of 93% on eight known objects with adversarial geometry and is 3x faster than registering point clouds to a precomputed dataset of objects and indexing grasps. The GQ-CNN is also the highest performing method on a dataset of ten novel household objects, with zero false positives out of 29 grasps classified as robust and a 1.5x higher success rate than a point cloud registration method.
Article
Full-text available
Soft robots are one of the most significant recent evolutions in robotics. They rely on compliant physical structures purposefully designed to embody desired characteristics. Since their introduction, they have shown remarkable applicability in overcoming their rigid counterparts in such areas as interaction with humans, adaptability, energy efficiency, and maximization of peak performance. Nonetheless, we believe that research on novel soft robot applications is still slowed by the difficulty in obtaining or developing a working soft robot structure to explore novel applications.
Conference Paper
Full-text available
Robots that autonomously manipulate objects within warehouses have the potential to shorten the package delivery time and improve the efficiency of the e-commerce industry. In this paper, we present a robotic system that is capable of both picking and placing general objects in warehouse scenarios. Given a target object, the robot autonomously detects it from a shelf or a table and estimates its full 6D pose. With this pose information, the robot picks the object using its gripper, and then places it into a container or at a specified location. We describe our pick-and-place system in detail while highlighting our design principles for the warehouse settings, including the perception method that leverages knowledge about its workspace, three grippers designed to handle a large variety of different objects in terms of shape, weight and material, and grasp planning in cluttered scenarios. We also present extensive experiments to evaluate the performance of our picking system and demonstrate that the robot is competent to accomplish various tasks in warehouse settings, such as picking a target item from a tight space, grasping different objects from the shelf, and performing pick-and-place tasks on the table.
Article
E-commerce retailers face the challenge to assemble large numbers of time-critical picking orders each consisting of just a few order lines with low order quantities. Traditional picker-to-parts warehouses are often ill-suited for these prerequisites, so that automated warehousing systems (e.g., automated picking workstations, robots, and AGV-assisted order picking systems) are applied and organizational adaptions (e.g., mixed-shelves storage, dynamic order processing, and batching, zoning and sorting systems) are made in this branch of industry. This paper is dedicated to these warehousing systems especially suited for e-commerce retailers. We discuss suited systems, survey the relevant literature, and define future research needs.
Article
This paper presents an overview of the inaugural Amazon Picking Challenge along with a summary of a survey conducted among the 26 participating teams. The challenge goal was to design an autonomous robot to pick items from a warehouse shelf. This task is currently performed by human workers, and there is hope that robots can someday help increase efficiency and throughput while lowering cost. We report on a 28-question survey posed to the teams to learn about each team's background, mechanism design, perception apparatus, planning, and control approach. We identify trends in this data, correlate it with each team's success in the competition, and discuss observations and lessons learned based on survey results and the authors' personal experiences during the challenge.