Kinect V2 Programming

NtKinect: Kinect V2 C++ Programming with OpenCV on Windows10

We created a class library NtKinect (this site, github) that makes it easy to program Kinect V2 with C++ and released it as Open Source of MIT License. Now, we will explain how to use it.

NtKinect supports multi-threading and is DLLable. So you can easily write program that uses Kinect V2 functions, and use it in other languages/environments, such as Unity for example.

• NtKinect: How to get RGB camera image with Kinect V2 (Fundamental Settings)
• NtKinect: How to get Depth image with Kinect V2
• NtKinect: How to make RGB image correspond to Depth image with Kinect V2
• NtKinect: How to get BodyIndex image with Kinect V2
• NtKinect: How to extract human images using Depth and BodyIndex image with Kinect V2
• NtKinect: How to get Infrared image with Kinect V2
• NtKinect: How to recognize human skeleton with Kinect V2
• NtKinect: How to recognize joint positions and palm state with Kinect V2
• NtKinect: How to recognize human face with Kinect V2 in ColorSpace coordinate system
• NtKinect: How to recognize human face with Kinect V2 in ColorSpace coordinate system (2)
• NtKinect: How to recognize human face with Kinect V2 in DepthSpace coordinate system
• NtKinect: How to recognize detailed face information (HDFace) with Kinect V2
• NtKinect: How to recognize both of face and HDFace with Kinect V2
• NtKinect: How to get and record audio with Kinect V2
• NtKinect: How to get audio beam direction with Kinect V2
• NtKinect: How to detect talking skeleton with Kinect V2
• NtKinect: How to recognize speech with Kinect V2
• NtKinect: How to recognize Kinect V2 audio by Cloud Speech API of Google Cloud Platform
• NtKinect: How to record video with Kinect V2
• NtKinect: BallFall game with Kinect V2
• NtKinect: How to recognize gesture with Kinect V2
• NtKinect: How to run Kinect V2 in a multi-thread environment
• NtKinect: How to make Kinect V2 program as DLL and use it from Unity
• NtKinect: How to make Kinect V2 face recognition as DLL and use it from Unity
• NtKinect: How to make Kinect V2 speech recognition as DLL and use it from Unity
• NtKinect: How to make Kinect v2 gesture recognition as DLL and use it from Unity
• NtKinect: How to make Kinect v2 skeleton recognition as DLL and use it from Unity
• NtKinect: How to make Unity's Humanoid take the same pose as Kinect V2's skeleton recognized in real time
• NtKinect: [Application] How to measure breath cycle with Kinect V2

Dynamic Linking Library (DLL)

• NtKinectDLL (DLL for NtKinect) and Wrappers (C#, Python)

Notice

• We will present about "Gesture Recognition with non-contact sensor for Natural User Interface in the COVID-19 era" at the international conference of HICSS-54, Jan/05-08/2021. This presentation is an application of this NtKinect library. (Jan/08/2021)

Yoshihisa Nitta, Himawari Ichikawa, Yuko Murayama: Gesture Recognition with non-contact sensor for Natural User Interface in the COVID-19 era, Proceedings of the 54th Hawaii International Conference on System Sciences (HICSS-54), 2021.

• I uploaded some projects with Visual Studio 2019 and OpenCV 4.1.1 to github.(Aug/31/2019)
https://github.com/YoshihisaNitta/NtKinect/tree/master/VS2019_CV411
• We will present about "Privacy-aware Remote Monitoring System by Skeleton Recognition" at the international conference of HICSS-52, Jan/08-11/2019. This presentation is an application of this NtKinect and NtKinectDLL library. (Sep/04/2018, Jan/20/2019 added)

Yoshihisa Nitta, Yuko Murayama: Privacy-aware Remote Monitoring System by Skeleton Recognition, Proceedings of the 52th Hawaii International Conference on System Sciences (HICSS-52), 2019.

• We made the presentation about "Software Support for Skeleton Recognition and Monitoring People with Privacy" at the international conference of HICSS-51, Jan/03-06/2018. This presentation was for the privacy oriented watching system developed with the NtKinect library. (Sep/12/2017, Jan/20/2018 added)

Yoshihisa Nitta, Yuko Murayama: Software Support for Skeleton Recognition and Monitoring People with Privacy, Proceedings of the 51th Hawaii International Conference on System Sciences (HICSS-51), p.200-206, 2018.

• We changed the compiler from "Visual Studio Professional 2015" to "Visual Studio Community 2017" in the sample. We also changed the OpenCV version from 3.1 to 3.3 and its install drive from "D:" to "C:". (Aug/30/2017)
• We have developed and released the NtKinectDLL which is a DLL that can use almost all the functions of NtKinect. In addition, We released NtUnity.cs that uses this DLL in Unity. You can see both from here.
• We announced this library's design policy at the 172-th conference of SIG Human-Computer Interaction, IPSJ.
You may refer NtKinect with the following description.

Yoshihsa Nitta: NtKinect : C++ Class Library for Kinect V2, 172-th conference, SIG Human Computer Interaction, IPSJ, 2017.

• We wrote an explanatory article on how to make Kinect V2 gesture recognition as DLL and use it in Unity. (Nov/13/2016)。
• With the recognition of detailed face information (HDFace), we have made it possible to select an option to generate individual's face model. However, it is unstable, so please treat it as experimental. (Nov/04/2016)
• When referring NtKinect, we recommend the following description. (Oct/30/2016)

Yoshihisa Nitta: NtKinect - Kinect V2 C++ Programming with OpenCV on Windows10 http://nw.tsuda.ac.jp/lec/kinect2/

• We fixed the download link of the VisualStudio project to craete DLL files for speech recognition. (Oct/18/2016)
• We fixed some sample projects in which some windows were displayed on the second screen. Also, we fixed the download link of Unity projects of speech recognition. (Sep/27/2016)
• We added an explanatory article on how to make Kinect V2 speech recognition as DLL and use it in Unity. (Sep/12/2016)
• We added an explanatory article on how to make Kinect V2 face orientation recognition as DLL and use it in Unity. (Aug/25/2016)