Grabbable rigidbodies

In this guide, I’ll walk you through creating grabbable rigidbody objects with Normcore.

Before we get started, you’ll want to download this project we put together for this guide. It includes a basic environment with some grabbable cubes, and single player VR support.

Go ahead and open the project - at the time of writing this guide, we’re using 2019.1.14, so you’ll need that version or later.

A quick overview of the project

Go ahead and open the main scene under Main.unity in your project inspector. The scene contain a basic environment with a floor and skybox, as well as some colored cubes resting on a table. The cubes may not look like much, but they are our grabbable rigidbodies!

If you look at the inspector for any of the coloured cubes by selecting one in the Hierarchy under Grabbable Cubes > Cube, you’ll notice each one contains Cube component–the first half of our logic.

Our scene also contains two Cube Grabber objects - one nested under each player hand. You can see them for yourself by selecting one under Player > Left/Right Hand > Cube Grabber in the Hierarchy. You’ll see each one has a matching Cube Grabber component added to it – this is where the second half of our logic lives. This Cube Grabber component interacts with the Cube component living on each Cube, and vice-versa.

Now that we’ve taken a look at the project hierarchy, let’s dive into how the Cube Grabber and Cube components interact with each other.


Below is the script you’ll have in your project under CubeGrabber.cs. Feel free to skim through it, and we’ll break it down step-by-step after the excerpt. If you’re following along in the editor, open the file in Visual Studio to reference.

    // CubeGrabber.cs
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR;

public class CubeGrabber : MonoBehaviour {
    //// Class ////
    // A list of all grabbable objects.
    private static HashSet<Cube> _cubes = new HashSet<Cube>();

    public static void RegisterCube(Cube cube) {

    public static void UnregisterCube(Cube cube) {

    //// Instance ////

    // Keep track of which hand we want to watch.
    private enum Hand { LeftHand, RightHand };
    [SerializeField] private Hand _hand;

    // Whether or not we're grabbing.
    private bool _triggerPressed;

    // Information on the current cube we're holding & controller velocity
    private Cube        _activeCube;
    private Vector3     _grabPositionOffset;
    private Quaternion  _grabRotationOffset;

    // Current controller pose data. Updated in UpdatePose().
    private Vector3     _lastValidControllerVelocity;
    private Vector3     _lastValidControllerAngularVelocity;

    void Update() {
        XRNode handNode = _hand == Hand.LeftHand ? XRNode.LeftHand : XRNode.RightHand;
        string trigger  = _hand == Hand.LeftHand ? "Left Trigger" : "Right Trigger";

        // Get velocity & angular velocity of the hand.
        bool handIsTracking = UpdatePose(handNode, ref _lastValidControllerVelocity, ref _lastValidControllerAngularVelocity);

        // Determine if we're pressing the trigger far enough to activate grabbing.
        bool triggerPressed = Input.GetAxisRaw(trigger) > 0.8f;

        bool triggerBegan = triggerPressed && !_triggerPressed;
        bool triggerActive = triggerPressed;
        bool triggerEnded = !triggerPressed && _triggerPressed;

        // If we start holding the trigger and aren't already holding a cube, start grabbing it.
        if (triggerBegan && _activeCube == null) {
            Cube cubeToGrab = GetClosestGrabbableCube();

            // Found a valid cube, let's grab it!
            if (cubeToGrab != null) {
                _activeCube = cubeToGrab;

                // Calculate grab offset.
                _grabPositionOffset = transform.InverseTransformPoint(_activeCube.transform.position);
                _grabRotationOffset = Quaternion.Inverse(transform.rotation) * _activeCube.transform.rotation;

        // If we're currently holding a cube, position it.
        if (triggerActive && _activeCube != null) {
            // Position the cube

        // If we let go of the trigger with an active cube, drop it.
        if (triggerEnded && _activeCube != null) {
            // Calculate cube throw velocity. Take into account the offset from the controller.
            Vector3 cubeVelocity = _lastValidControllerVelocity + Vector3.Cross(_lastValidControllerAngularVelocity, transform.rotation * _grabPositionOffset);
            Vector3 cubeAngularVelocity = _lastValidControllerAngularVelocity;

            // Throw cube.
            _activeCube.EndGrabbing(cubeVelocity, cubeAngularVelocity);
            _activeCube = null;

        // Update cached trigger value.
        _triggerPressed = triggerPressed;

    private Cube GetClosestGrabbableCube() {
        Cube cubeToGrab = null;
        float cubeToGrabDistance = float.MaxValue;
        foreach (Cube cube in _cubes) {
            // Skip cubes that have already been grabbed.
            if (cube.grabbed) continue;

            float distance = cube.DistanceToCubeSurface(transform.position);

            // Skip if we're not colliding with the cube.
            if (distance > 0.0f) continue;

            // Replace cubeToGrab if it's closer than any previously seen cube.
            if (distance < cubeToGrabDistance) {
                cubeToGrab = cube;
                cubeToGrabDistance = distance;

        return cubeToGrab;

    // Position the active cube.
    private void PositionActiveCube() {
        // Calculate the cube position, including delta grab offset.
        Vector3 cubePosition = transform.TransformPoint(_grabPositionOffset);
        Quaternion cubeRotation = transform.rotation * _grabRotationOffset;

        // Position the cube.
        _activeCube.PositionCube(cubePosition, cubeRotation);

    private static bool UpdatePose(XRNode node, ref Vector3 velocity, ref Vector3 angularVelocity) {
        // Update velocity & angular velocity with the given XRNode via XRNodeState
        List<XRNodeState> nodeStates = new List<XRNodeState>();

        foreach (XRNodeState nodeState in nodeStates) {
            if (nodeState.nodeType == node) {
                // Matches the hand we're looking for - let's grab its velocity.
                Vector3 nodeVelocity;
                Vector3 nodeAngularVelocity;

                bool gotVelocity        = nodeState.TryGetVelocity(out nodeVelocity);
                bool gotAngularVelocity = nodeState.TryGetAngularVelocity(out nodeAngularVelocity);

                if (gotVelocity)
                    velocity = nodeVelocity;
                if (gotAngularVelocity)
                    angularVelocity = nodeAngularVelocity;

                return gotVelocity;

        return false;

So firstly, our CubeGrabber implementation defines some static members, used to create a single registry of grabbable cubes across both grabbers. These members are:

  • _cubes, a HashSet<Cube> property for holding the references to each registered cube
  • RegisterCube(Cube cube) and UnregisterCube(Cube cube), functions called by each Cube to register or unregister itself from the grabbers

After our class members are the instance members, for holding state related to each independent grabber.

  • an internal Hand enum with types LeftHand and RightHand for representing either the left or right hand
  • _hand for setting which Hand the grabber belongs to
  • _triggerPressed for denoting whether the appropriate trigger is currently being pressed
  • _activeCube for storing the currently grabbed Cube, if one is being held
  • _grabPositionOffset and _grabRotationOffset for storing the position and rotation offset of the _activeCube if there is one

Finally, our instance methods make up the rest of the file, defining the logic for checking to see if we need to start grabbing cubes, as well as position the currently grabbed Cube each frame (if there is one).

  • An UpdatePose method: used to grab the current velocity & angular velocity of the appropriate controller with the XRNodeState APIs
  • GetClosestGrabbableCube(): returns the closest valid grabbable cube relative to the respective grabber position
  • PositionActiveCube(): used to position the cube being actively grabbed, if there is one
  • Our Update loop: determine on each frame if we should start, continue, or stop grabbing a cube. Uses our UpdatePose, GetClosestGrabbableCube, and PositionActiveCube methods

That about sums up the CubeGrabber implementation. Now, let’s take a quick look at the Cube implementation.


Below is our Cube script - like before, feel free to follow along in Visual Studio for reference when reading the member descriptions.

    // Cube.cs
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Cube : MonoBehaviour {
    [SerializeField] private float _grabRadius = 0.25f;

    public bool grabbed { get; private set; }

    private Rigidbody _rigidbody;

    // Unity events
    private void Awake() {
        _rigidbody = GetComponent<Rigidbody>();

    private void OnEnable() {

    private void OnDisable() {

    // Used to verify we're actually colliding with the cube at the given grabPosition.
    public float DistanceToCubeSurface(Vector3 grabPosition) {
        return Vector3.Distance(grabPosition, transform.position) - _grabRadius;

    // Grab events
    public void BeginGrabbing() {
        // Mark as grabbed.
        grabbed = true;

        // Mark the rigidbody as kinematic.
        _rigidbody.isKinematic = true;

    public void PositionCube(Vector3 position, Quaternion rotation) {
        // Move rigidbody

    public void EndGrabbing(Vector3 velocity, Vector3 angularVelocity) {
        // Mark the rigidbody as non-kinematic
        _rigidbody.isKinematic = false;
        _rigidbody.velocity = velocity;
        _rigidbody.angularVelocity = angularVelocity;

        // Mark as free to grab again.
        grabbed = false;

Our Cube class has a few key pieces of functionality. Firstly, you’ll see there’s a _grabRadius that you can configure for determining an appropriate radius for your grabbable objects. Feel free to adjust this if you’d like, but we’ll keep it at 0.25.

On Awake() and OnEnable(), each Cube registers with the CubeGrabbers in our scene, by calling the static method CubeGrabber.RegisterCube and passing itself. It also caches a reference to its own Rigidbody as _rigidbody, so it can disable or enable gravity when grabbing begins or ends.

That sums up the Cube implementation - it’s much more straightforward than the Cube Grabber, but equally as important! Feel free to hit Play and put on your VR headset to pick up some cubes and try it out.

Check out that sweet cube-grabbing action. Nice.

Multiplayer integration

To get this working with Normcore, we’ll only need a few additions. First, of course, be sure to download the latest version of Normcore from the Dashboard, and go ahead and import it into your project.

Import the latest version of Normcore into your project.

With Normcore imported, head to the Normal > Examples > VR Player folder in your project Hierarchy and drag a Realtime + VR Player prefab into your scene. Paste in an app key from the Dashboard, and set a room name. While we’re here, we can hook up our local player avatar to Realtime Avatar Manager, by dragging in the root Player, Head, Left Hand, and Right Hand objects into the appropriate fields.

Add a Realtime + VR Player prefab into your scene, and hook up your local Player avatar to the Realtime Avatar Manager.

With Normcore set up with our player, we’ll need to do one last thing and network our cubes. We can do this by simply adding a Realtime Transform component to each Cube. This will add both the Realtime Transform and Realtime View components to the Cube, and Normcore will automatically synchronize the transform of the cubes for all players.

Add a Realtime Transform component to each of your grabbable cubes.

Now, we need to make one last modification before we can call the project complete. With a Realtime Transform component on each Cube, we need to be sure to request ownership of the transform when grabbing it. Let’s do so now with a few additions to Cube.cs: caching a reference of our RealtimeTransform, as well as calling RequestOwnership() in BeginGrabbing.

    // Cube.cs
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Normal.Realtime;

public Class Cube : MonoBehaviour {
    // ...

    private Rigidbody         _rigidbody;
    private RealtimeTransform _realtimeTransform;

    // Unity Events
    private void Awake() {
        _rigidbody         = GetComponent<Rigidbody>();
        _realtimeTransform = GetComponent<RealtimeTransform>();

    // ...

    public void BeginGrabbing() {
        // ...

        // Request ownership from the transform.

Awesome. With that settled, our cubes should be properly networked. Our implementation works the same as it does without Normcore: when we grab a cube, our CubeGrabber will set the Cube rigidbody to kinematic, allowing our user to position the cube freely. Then, when the user lets go of the Cube, the rigidbody is marked as non-kinematic again, allowing the Cube to fall and come to rest. The only Normcore changes necessary were adding RealtimeTransform to each Cube, and requesting ownership of the appropriate RealtimeTransform on grab.

Feel free to build & run the project and send it to a friend - you should both be able to grab the cubes in VR and toss them around to one another. It’s pretty basic in its current form, but does a great job at illustrating how to handle user interaction with networked objects in Normcore!