To grasp or not to grasp: an end-to-end deep-learning approach for predicting grasping failures

This paper tackles the challenge of predicting
grasp failures in soft hands before they happen, by combining
deep learning with a sensing strategy based on distributed
Inertial Measurement Units. We propose two neural architectures, which we implemented and tested with an articulated soft
hand - the Pisa/IIT SoftHand - and a continuously deformable
soft hand - the RBO Hand. The first architecture (Classifier)
implements a-posteriori detection of the failure event, serving
as a test-bench to assess the possibility of extracting failure
information from the discussed input signals. This network
reaches up to 100% of accuracy within our experimental
validation. Motivated by these results, we introduce a second
architecture (Predictor), which is the main contribution of the
paper. This network works on-line and takes as input a multidimensional continuum stream of raw signals coming from
the Inertial Measurement Units. The network is trained to
predict the occurrence in the near future of a failure event.
The Predictor detects 100% of failures with both hands, with
the detection happening on average 1.96 seconds before the
actual failing occurs