NtKinectDLL and NtUnity: Tutorial:

Use the Audio Beam Direction on Unity acquired with Kinect V2

Prerequisite knowledge

• Using various functions of Kinect V2 on Unity (Fundamental Settings)
• NtKinect: How to get audio beam direction with Kinect V2

Acquire the audio beam direction with Kinect V2 and use it in Unity

1. I will use the Unity project created in "Using NtKinectDLL in Unity (Fundamental Settings)&auot;. Download UnityDLL01.zip and expand it. Also change the name of the expanded folder to "UnityDLL04/".
2. Create a new scene and save it under "Assets/Scenes/" as "Sample04.unity".

File -> New Scene

File -> Save Scene as ... -> Sample04.unity

3. Create a Folder "Assets/Materials/" in the project, and create 4 Materials in it.

Change the name and Albedo of each Material as follows.

Name	Albedo
	R	G	B	A
Red	255	0	0	255
Green	0	255	0	255
Blue	0	0	255	255
Ground	214	164	0	255








4. Create 3 Cylinders in Herarchy. Change each name and set Transform and Material.
Above Menu -> GameObject -> 3D Object -> Cylinder

Name	Position			Rotation			Scale			Material
	X	Y	Z	X	Y	Z	X	Y	Z
X-axis	0	0	0	0	0	90	0.1	10	0.1	Red
Y-axis	0	0	0	0	0	0	0.1	10	0.1	Green
Z-axis	0	0	0	90	0	0	0.1	10	0.1	Blue





5. Create a Plane in Hierarchy. Change its name as "Ground" and set the Transfrom and Material as follows.
Above Menu -> GameObject -> 3D Object -> Plane

Name	Position			Rotation			Scale			Material
	X	Y	Z	X	Y	Z	X	Y	Z
Ground	0	0	0	0	0	0	5	5	5	Ground





6. Create a Sphere in Hierarchy. Change its name as "Marker" and set the Transfrom. Leave its Material as white as default.
above menu -> GameObject -> 3D Object -> Sphere

Name	Position			Rotation			Scale			Material
	X	Y	Z	X	Y	Z	X	Y	Z
Marker	0	0.1	0	0	0	0	1	0.1	1	White (Default-Material)





7. Create new C# script under "Assets/Scripts/" in the Project window.

from above menu -> "Assets" -> "Create" -> "C# Script" -> Rename as "Sample04"

Sample04.cs

using System.Collections; using System.Collections.Generic; using UnityEngine; using System; public class Sample04 : MonoBehaviour { public GameObject target; public float radius = 2.0f; NtUnity.Kinect nt; void Start () { nt = new NtUnity.Kinect(); } void Update () { nt.setAudio(false); if (nt.beamAngleConfidence < 0.5) return; double theta = nt.beamAngle; double x = radius * Math.Sin(theta); float y = target.transform.position.y; double z = radius * (-Math.Cos(theta)); target.transform.position = new Vector3((float)x,y,(float)z); } }

8. Place "Empty Object" in Hierarchy, and change its name to "GameController". Add "Sample04.cs" as a component to it, and set "Marker" in Hierarchy to the "Target" item.

above menu -> GameObject -> Create Empty











9. With the "Main Camera" selected in Hierarchy, change its Transform to Position (x,y,z) = (0,1,-3), Rotateion=(20,0,0).
10. When executed, the Marker moves so that there is as sound source on the line connecting the origin and the Marker.
Runtime Video UnityDLL02.mp4




11. The sample project of Unity is UnityDLL04.zip。
12. By the way, the movie at the head of this web page is obtained by adding the following 4 lines to the top of the Update() function of "Sample04.cs". For the privacy, face recognition is performed and face recognition is made to hide the face.

     nt.setRGB();
     nt.setSkeleton();
     nt.setFace();
     nt.imshowBlack();