The future is now

Moving from structured to unstructured environments is a clear trend in robotics. For example, a state-of-the-art problem for warehouse robots is manipulating packages with various shapes without models, in contrast to manipulating objects with predefined models in more controlled environments such as car factories and laboratories.

One of the ultimate questions in this line with even stronger technological and financial influences would be: can we make affordable personal robots that can perform fine manipulation of objects in unstructured human environments such as our homes and businesses?

I jumped onto this seemingly unreachable problem and have achieved promising results using three simple components: an inexpensive tactile sensor, a common collaborative robot, and model-free deep reinforcement learning.

Challenges in Unstructured Human Environments for Robot Manipulation

Variations - hard to obtain models:

  • Hindering traditional model-based analytical approaches. They cannot work without high-quality models.

  • Make the "guess" - online estimation is unavoidable, which is accompanied by uncertainty.

Uncertainty - "it's not as expected":

  • Object properties may not be as expected, e.g., the uncertainty in object shape and weight distribution.

  • Contact properties may differ from expectations, e.g., friction may change with humidity levels.

Dexterity - more DOF at contacts:

  • Current studies in robot manipulation relies on fixed contacts such as those in stable static grasping.

  • Future robots should master advanced contact interactions such as slipping and rolling.

The Approach: Robot Learning for Fine Manipulation

Fine manipulation is beyond dexterous manipulation.

Similarity: Both allow high degrees of freedom (DOF) at contacts.

Difference: Fine manipulation is ready for unstructured human environments thanks to the following features:

  • The mastering of contact interactions - like humans, a fine manipulation skill can use slipping to relocate contacts and use sticking or rolling contacts to manipulate objects.

  • Robust to variations and uncertainties - when the properties of objects and conditions of contacts are not as expected, a fine manipulation skill can adjust contact locations and transition between contact modes (the combination of contact interactions of all contacts) to manipulate the objects properly.

Below are the key ideas of the methodology for achieving fine manipulation skills:

End-to-end Model-free Deep Reinforcement Learning:

  • It can map raw perception directly to actions, integrating and simplifying processes including modeling, estimation, planning, and control.

Information from Interaction:

  • Important information may not be readily available in unstructured environments, and some can only be unveiled by making contact and interacting with the object.

Active Compliance

  • Enables reinforcement learning of fine manipulation skills - the robot may explore with maintained contact.

  • The robot may actively use and transition between contact modes.

The Setup:

Inexpensive, information-poor tactile sensor:

  • Commercially available, it only measures the magnitude of the normal contact force, not the contact location or the normal direction.

  • I achieved model-free active compliance with this sensor on a regular collaborative robot.

Delicate Contact.mp4

Crossed the reality gap:

  • The reinforcement learning agents can be trained in simulation and deployed in the real world.

  • This fact differentiates my approach from other studies that look fancy but only work in simulation. This approach is a meaningful and promising start.

Demonstrations - Primitive Fine Manipulation Skills

(Objects were tracked by motion capture)

  • The robot tries moving a small box (it's center) from the top left corner to the middle with a sticking contact.

  • The contact mode may change during execution due to uneven surface or location-dependent friction. The robot notices the changing condition and adapts to it.

  • Without retraining, the robot can immediately manipulate a larger and heavier box.

  • The contact condition is imperfect due to the gaps between the black tapes. When contact breakage occurs, the robot can reestablish the contact and bring the box to the goal.

  • The robot is trying to tilt a box. Due to contact friction and the robot's force limit, a contact location too far away from the right edge of the box can be infeasible for tilting.

  • In this experiment, the initial contact is in the feasible region, and the robot tilts the object right away without hesitation.

  • If the initial contact is infeasible, the robot can relocate the contact to the feasible location with slipping contact and tilt the object.

  • This process is done autonomously without further training or fine tuning.

  • The tilting skill can generalize to various real-world scenarios. This video is a compilation of the robot manipulating three boxes with different shapes and weights.

  • In the 1st row, the bottom contact between the box and the table has high friction, leading to feasible initial contacts. In the 2nd row, the bottom contact has low friction; thus, the initial contacts are infeasible. The robot was able to adapt to all variations without additional training.

To demonstrate the robustness of the tilting skill, here's a compilation of ten consecutive executions.

Affordable personal robots with fine manipulation skills and human-like adaptability is possible.