Imran's personal blog

February 26, 2017

My Unity/Daydream VR notes

Filed under: Uncategorized — ipeerbhai @ 4:13 am

Background

I’m over at the Seattle VR Hackaton sponsored by AT&T over the weekend, and decided to build a Daydream version of our DreamHUD hackathon project.  It quickly became apparent that this wasn’t going to work.  So, instead I decided to try and figure out how to implement Unity Daydream VR in any capacity at all.  I talked to 6 other developers here at the VR Hackathon, and none — I repeat none — got even “hello world” to boot on their Pixel/Daydreams using Unity and the Google VR SDK.  Almost all ended up with a black screen or compile failures.  I’m the only one who got something to build and deploy, but I’ve had no luck in getting and keeping a daydream app up and running with Unity.  I’m hoping my notes help others ( and myself ) get a working daydream VR app in the future.

Example Source Code:

I put the example source code on GitHub as a public project.

You can find the repo with bugs here:

https://github.com/ipeerbhai/DayDream101

then rebuilt it here with fewer bugs:

https://github.com/ipeerbhai/UnityAndroid101

Main issues

The main issues in getting Unity + Daydream working are:

  1. Install order seems to matter.  Installing things out of order results in “black screen of death” deployments.
  2. The controller emulator doesn’t work.  With some probing with adb, I was able to, a few weeks later, figure out how to get it to work.  Please see the troubleshooting section at the end of this blog post.
  3. The GvrEventSystem craps out during Play on Windows with the controller emulator.  As in the event pump either crashes the Unity editor, or the events just stop firing.
  4. Deploying to Cardboard results in a black screen.
  5. Poor documentation.  I thought MS was bad at MSDN docs — but they’re heaven compared to Google’s docs.  No example uses of any of their classes.  Even their own demos crash/blackscreen, so we can’t attach breakpoints and debug our way to figure out their APIs.

Notes

Installation:

Start with the Google instructions here:

https://developers.google.com/vr/unity/get-started

Here’s a few tricks I learned from failures along the way.

  • Make sure you have Android studio 2.2.3 or newer before you start install of Unity/JDKs.
  • For daydream controller emulator support in the Unity player, you must put ADB.exe in your system path variable.
  • open a cmd or shell window and run “adb devices” before starting Unity.  Unity’s player won’t be able to debug controller issues if you don’t.
  • Make sure you have Unity 5.6 or newer installed.
  • You must install Java SE JDK 1.8 along with Android SDK 24+ for daydream to work.  You can install the Android SDK from android studio, and the JDK from Unity.
    • Android Studio for SDK:  Click The Tools menu –> android –> SDK Manager
    • Unity for JDK: Edit –> Preferences… –>External Tools.  Click the little download button next the the JDK textbox.
  • Import the Google VR SDK unitypackage as the *very first thing* in the project!  Making a scene first, then importing the SDK will cause really hard to debug crashes.
  • On Windows, installing the Unity Tools for Visual Studio really makes script development in C# easier.
  • If you get a lot of controller object errors while running the player, stop the player and restart the scene.
  • Order operations really seems to matter.  Weird crashes and hard to debug issues seem to resolve if you change the order in which you install things or start unity.

After you’ve set the settings from the Google VR Guide, your main camera is now a VR stereo camera.  You can now create a scene.

Editor UI stuff:

On the upper right corner of the scene is a cube with some cones coming out of it.  That’s to control the scene camera in development mode.  Click the cube to enable right-click rotation, and click the cones to “look down” that axis towards the origin.

Questions and Answers:

How do I get a reference to a GameObject from a “Master” script attached to another gameobject?

Example Answer:  Create a GameObject in the Unity editor ( I created a cube, named it “Cube1” ).  In the master script’s Update function, I did this:

void Update () {
var Cube1 = GameObject.Find(“Cube1”);
}

How do I rotate this cube?

var cubeTransform = Cube1.transform;
cubeTransform.Rotate(Vector3.up, 10f * Time.deltaTime); // Time.detaTime is a unity provided static float that represents the time in seconds between calls to the update function.  The parameters seem to be a global-cooridinate axis ( Vector3.up is a unit vector [0,1,0] )and an arc-second.

What’s Unity’s cooridinate system?

Unity has X moving left/right, Z moving forward back, and Y moving up and down.  This is “Left hand rule” with the thumb as X, the index pointing up as Y, and the middle pointing forward as Z.

What’s the difference between a terrain and a Plane?
Digested from https://www.youtube.com/watch?v=Oc3odBj-jFA, and unity docs.

Terrains are defaulted to 500 x 500 meters in X and Z, with their “lower left”  set to 0, 0,  0.  You can deform them, and there’s a default material renderer with property settings the can mimic different types of ground ( like grass, sand, concrete. )  Planes are smaller, can’t deform, and don’t have a default texture.  Here’s a good shot of the inspector for terrain.

How do I make a “grassland with trees” texture onto the terrain?

  1. Import the Standard asset package to get some basic textures.  You can skip this step if you already have the texture you want.
    1. C:\Program Files\Unity 5.6.0b9\Editor\Standard Assets
  2. Select the PaintBrush tool in the terrain inspector.
  3. Click the “Edit Textures” button.
  4. Select “Add Texture”
    1. You can either click the “select” button and pick the asset in a flat list of textures.
    2. You can “drag and drop” the asset icon
    3. I picked, “GrassHillAlbedo.psd”
  5. Add the trees.
    1. Select the tree terrain “brush”.
    2. Click “Edit Trees…”
    3. Click add
      1. Pick one of the standard trees.
      2. Or, you can pick a tree you pre-modeled from the Unity tree modeler.

How do I make a sphere and Bounce it infinitely?
Digested from this video:
https://unity3d.com/learn/tutorials/projects/roll-ball-tutorial/moving-player?playlist=17141

  1. In the unity editor:
    1. Make a sphere of radius = 1 meter, position = 0, 0.5, 0.
    2. Attach a physics rigidbody component to it.
  2. In the MasterScript ( Or in the script for the object — I want to keep everything in one master script/gameobject, type this code in:
  3. // Update is called once per physics engine call, usually before update
    private void FixedUpdate()
    {
    // update all the forces we want to…
    var mySphere = GameObject.Find(“Sphere”);
    var theRigidBody = mySphere.GetComponent();
    if (mySphere.transform.position.y < 0.51)
    theRigidBody.AddForce(0, 300, 0, ForceMode.Acceleration);
    }

How do I enable Controller Support?
Digested from:
https://developers.google.com/vr/unity/controller-support

  1. Create an empty GameObject and name it Player.
  2. Set the position of the Player object to (0,1.6,0).
  3. Place the Main Camera underneath the Player object at (0,0,0).
  4. Place GvrControllerPointer underneath the Player object at (0,0,0).
  5. Set the position of the Main Camera to be (0,0,0).
  6. Add GvrViewerMain to the scene, located under GoogleVR/Prefabs.
  7. Add GvrControllerMain to the scene, located under GoogleVR/Prefabs/Controller.
  8. Add GvrEventSystem to the scene, located under GoogleVR/Prefabs/UI.

At the end of this, you’ll have a “laser pointer” on your “right hand” in your app.

How do I know what the DayDream controller is pointing at?

Digested from https://www.youtube.com/watch?v=l9OfmWnqR0M

There are two ways I’ve found to do this.

Method 1:  Use raycasting.

  1. Get the controller’s position be treating the controller as any gameobject.
    1. GameObject controllerPointer = GameObject.Find(“GvrControllerPointer”);
      Transform controllerTransform = controllerPointer.transform;
      Vector3 pos = controllerTransform.position;
    2. Can also get positoin in one line of code:
      Vector3 controllerPosition = GameObject.Find(“GvrControllerPointer”).transform.position;
  2. Get the controller’s orientation and create a forward pointing vector from the orientation quaternion.
    1. Vector3 fwd = GvrController.Orientation * Vector3.forward;
  3. Use phyiscs.Raycast to see what the controller is pointing at.
    1. RaycastHit pointingAtWhat;
    2. Physics.Raycast(pos, fwd, out pointingAtWhat);

 

Sample code: ( compiled and verified )

void Update ()
{
// find the bouncing sphere from inside this central game object.
var MySphere = GameObject.Find(“Sphere”); // Can skip getting if this script component is attached to the GameObject that will be the target.
var MySphereTransform = MySphere.transform;

// find the controller and get its position.
var controllerPointer = GameObject.Find(“GvrControllerPointer”);
var controllerTransform = controllerPointer.transform;

// use the controller orientation quaternion and get a forward pointing vector from it, then raycast.
Vector3 fwd = GvrController.Orientation * Vector3.forward;
RaycastHit pointingAtWhat;
if (Physics.Raycast(controllerTransform.position, fwd, out pointingAtWhat) )
{
var theTextGameObject = GameObject.Find(“txtMainData”);
UnityEngine.UI.Text theTextComponent = theTextGameObject.GetComponent<UnityEngine.UI.Text>();
theTextComponent.text = “hit ” + pointingAtWhat.collider.name;
}
}

Method 2: Use the Unity event system as modified by Google.

Step 1:  Add the GVREventSystem script to your scene, and add GvrPointerPhysicsRaycaster to your main camera.

Step 2:  Inherit from IGVRPointerHoverHandler on any gameobjects you want to receive a notification that the controller is pointing at like this:

public class myGameObject : MonoBehaviour, IGvrPointerHoverHandler {

public void OnGvrPointerHover(PointerEventData eventData) {

// myThing is now “hovered” by the controller pointer.  Do logic now.
// eventData contains a point of intersection between the ray and the object, along with a delta ( magic? )

}

// WARNING — this breaks in Unity 5.6.b11, but works through 5.6.b10.  Bug?

}

How do I rotate the camera around the world with the touchpad ?

Please note — don’t actually rotate the camera via the touchpad — it makes people sick fast.  This code is really only useful on the PC/control emulator to test input.

// Handling camera rotation with the controller touchpad needs these concepts:
// 1.The touchpad is an X/Y device.  (0,0) is top left.
//      X = 0 means “furthest left touch”.  X = 1 means “furthest right”.
// 2. We need a large “dead zone” around X = 0.5 to prevent jerky movement.
// 3. You cannot rotate the camera directly.  Google’s APIs reset any camera rotations.
//     Instead, put the camera in something, then rotate that something.

void Update()  // cut and paste, but modified, from working code.
{

float m_rotationSpeed = 10.0f; // normally a class member, put here for demo purposes.
float deadZone = 0.15f;
var player = GameObject.Find(“Player”); // the object containing the main camera.
if (GvrController.IsTouching)
{
if (GvrController.TouchPos.x < .5 – deadZone)
{
// Should be rotating left
player.transform.Rotate(0, -1 * Time.deltaTime * m_rotationSpeed, 0);
}
else if (GvrController.TouchPos.x > .5 + deadZone)
{
//Should be rotating right
player.transform.Rotate(0, 1 * Time.deltaTime * m_rotationSpeed, 0);
}
}
How do I hit the sphere with a “pool stick” on app button press?

In this scenario, we’re using the controller as a “pool stick” to hit the bouncing sphere and move it when the user pushes the app button.  Some learnings.

  1. AppButtonDown is only true for one frame — the frame when the user pushed the button.  This is a problem with a bouncing sphere, because the user may not have hit the bouncing ball when pushing the button.  Instead, we’ll use the boolean that’s always true, and add force as long as the button is down.
  2. GvrController does not expose position, so we have to use the GvrControllerPointer Prefab in Assets/GoogleVR/Prefabs/UI/GvrControllerPointer.prefab attached to a “Player” object.

private void FixedUpdate() // modified from working code.
{
// find the bouncing sphere from inside this central game object.
var mySphere = GameObject.Find(“Sphere”); // Can skip getting if this script component is attached to the GameObject that will be the target.
var sphereRigidBody = mySphere.GetComponent<Rigidbody>();
var MySphereTransform = mySphere.transform;

// find the controller and get its position.
var controllerPointer = GameObject.Find(“GvrControllerPointer”);
var controllerTransform = controllerPointer.transform;

// use the controller orientation quaternion and get a forward pointing vector from it, then raycast.
Vector3 fwd = GvrController.Orientation * Vector3.forward;
RaycastHit pointingAtWhat;
if (Physics.Raycast(controllerTransform.position, fwd, out pointingAtWhat))
{
if (GvrController.AppButton)
{
Vector3 forceToAdd = GvrController.Orientation * Vector3.forward * 100;
sphereRigidBody.AddForceAtPosition(forceToAdd, pointingAtWhat.point);
}
}

// update all the forces we want to…
if (mySphere.transform.position.y < 0.51)
sphereRigidBody.AddForce(0, 300, 0, ForceMode.Acceleration);
}

 

How do I display text in world view?

Digested from:
https://blogs.unity3d.com/2014/06/30/unity-4-6-new-ui-world-space-canvas/

Text in unity is rendered on a canvas.  This is a problem, because the GvrController class has a canvas.  So, if you create –> ui –> text, you’ll bind that text to your controller.  Instead, you have to make a canvas in world view, then adjust the scale from pixels to world units ( aka meters ).

  1. Create a new canvas using Create –> UI –>Canvas.  Give it a name.
  2. Select the Canvas In the Scene, then look at the Canvas Component in the Inspector.  Change the “Screen space overlay” to “World space”.
  3. The canvas is a gameobject — you can move it like you want.  But, don’t change the size property.  Instead, scale the canvas down to a reasonable size.
  4. With the canvas still selected, do create –> UI –> Text.  This will put text in the canvas.  Select the color and properties of the text in the inspector.

How do I start recording audio while the user has the app button down?

This turns out to require a new project and updating the Google SDK.  In the old SDK, Microphone permissions couldn’t be acquired, but now existing sample code works.

  1. Add an audio source to the component that is going to record.
  2. Decorate the GameObject source code for the recording component like this:
    1. [RequireComponent(typeof(AudioSource))]
  3. Add a private AudioSource to your GameObject derived class:
    1. private AudioSource microphoneAudioSource = null;
  4. Check for AppButtonDown in your Update function:
    1. if (GvrController.AppButtonDown) { // statements }
  5. Create a ringbuffer and call Microphone.Start like this:
    1. microphoneAudioSource.clip = Microphone.Start(null, true, 10, 16000);
      microphoneAudioSource.loop = true;
  6. Finish recording on AppButtonUp like so:
    1. if (GvrController.AppButtonUp) {
      int recordingPosition = Microphone.GetPosition(null); // do before calling end!
      Microphone.End(null);
      }
  7. AudioSource.clip will now contain the AudioClip.

 

Troubleshooting:

Problem: controller emulator won’t connect in the player.
Solution: Controller emulator device must be the first device that lists in adb devices.  This is a problem, in that some services on the host take port 5555 on up, and adb will see those sometimes.  Try running adb kill-server, then adb devices with your emulator phone attached.

Problem: Can’t install apk packages built on different PCs from same source code.
Solution: must uninstall the previous packages first.  You can use the android package manager (pm) to find the previously installed package, then run the uninstall command like so:
adb shell pm list packages
adb uninstall <com.xxx.yyy> (aka your old package)

Problem: Can’t get Mic input.
Solution: reinstall the GVR assets and build for non-VR Android target, run without VR, then re-enable Android VR target.  This seems to be caused by a bug in permission management in VR vs stock android.  Once your app has the perms, it keeps them.

 

Advertisements

3 Comments »

  1. Thanks for the instructions and tips! Even following those, however, I wasn’t able to get things working. It could be because I’m using an older phone (non-daydream), but I had all the options set to Cardboard. Regardless, I’m sure this will help others, as the GoogleVR SDK documentation leaves much to be desired.

    I just wanted to pay back the favor by linking you to the VRTK plugin, which makes it much easier to prototype controls in VR. It current supports SteamVR and OculusVR, while Daydream controls are in beta: http://vrtk.io/

    Cheers!

    Comment by Russell Alleen-Willems — February 28, 2017 @ 6:46 pm | Reply

  2. Thanks for the feedback — I also can’t get Cardboard apps written for both DayDream and Cardboard to do anything but blackscreen in Cardboard. I think it’s likely a bug in either Unity or the GVRSDK. Remeber. this is a Beta version of Unity and the GVRSDK. I’ll check out VRTK — thanks for the link!

    Comment by ipeerbhai — February 28, 2017 @ 8:26 pm | Reply

  3. If when attempting to build an Android package for Cardboard or Daydream on Unity and you experience an error saying “Unable to list target platforms. Please make sure the android sdk path is correct. See console for more details.”

    Locate your Android SDK folder and replace the “platform-tools” and “tools” folders with these older versions: [ tools_r25.2.2-windows.zip: https://dl.google.com/android/repository/tools_r25.2.2-windows.zip ] and [ platform-tools-latest-windows.zip: https://dl.google.com/android/repository/platform-tools-latest-windows.zip ]

    Comment by Aaron Butler — March 18, 2017 @ 8:55 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: