NtKinect: Kinect V2 C++ Programming with OpenCV on Windows10

How to get BodyIndex image with Kinect V2

2016.07.20: created by

2016.07.21: revised by

To Table of Contents

Prerequisite knowledge

NtKinect: How to get RGB camera image with Kinect V2 (Fundamental Settings)

BodyIndex image

BodyIndex images can be acquired with a resolution of 512 x 424 as well as Depth images. Up to six people can be distinguished at the same time.

In NtKinect, the obtained BodyIndex image is represented by uchar or cv::Vector3b for each pixel.

NtKinect

NtKinect's functions for BodyIndex image

type of return value型	function name	descriptions
void	setBodyIndex(bool raw = true)	Set BodyIndex image to the member variable "bodyIndexImage" If the argument raw is "true" or there is no argument, the type of each pixel is `uchar`. If the argument raw is "false", the type of each pixel is `cv::Vec3b`.

NtKinect

NtKinect's member variable for BodyIndex image

type variable name descriptions

cv::Mat

bodyIndexImage

BodyIndex image.
Up to six people can be detected at the same time, and BodyIndex itself is a number from 0 to 5 assigned to each detected person. The type of each pixel of the BodyIndex image may be uchar or cv::Vec3b. The coordinates of the image are positions in the DepthSpace coordinate system.

bodyIndex.cols --- Resolution for the horizontal direction of the image (= 512)
bodyIndex.rows --- Resolution for the vertical direction of the image (= 424)
bodyIndex.channels() --- Return 1 if the pixel is of type uchar, and 3 if cv::Vec3b.

In case of uchar data type

The value of each pixel is 0 to 5 in the pixel where a person is detected, and 255 in the other pixels.

bodyIndexImage.at<uchar>(y , x ) --- Access pixel at the (x , y ) coordinates of the image

    uchar pixel = bodyIndexImage.at<uchar>(y , x )

In case of cv::Vec3b data type

The value of each pixel is the RGB value corresponding to the bodyIndex number.

bodyIndexImage.at<cv::Vec3b>(y , x ) --- Access pixel in the (x , y ) coordinates of the image

    cv::Mat pixel = bodyIndexImage.at<cv::Vec3b>(y , x )

bodyIndex	cv::Vec3b
0	255	0	0
1	0	255	0
2	0	0	255
3	255	255	0
4	255	0	255
5	0	255	255

How to write program

Start using the Visual Studio's project KinectV2.zip of "NtKinect: How to get RGB camera image with Kinect V2 (Fundamental Settings)"
Change the contents of "main.cpp" as follows.

Call kinect.setBodyIndex() function to set bodyIndex data for each pixel in kinect.bodyIndexImage.

main.cpp

#include <iostream>
#include <sstream>

#include "NtKinect.h"

using namespace std;

void doJob() {
  NtKinect kinect;
  while (1) {
    kinect.setBodyIndex(false);
    cv::imshow("bodyIndex", kinect.bodyIndexImage);
    auto key = cv::waitKey(1);
    if (key == 'q') break;
  }
  cv::destroyAllWindows();
}

int main(int argc, char** argv) {
  try {
    doJob();
  } catch (exception &ex) {
    cout << ex.what() << endl;
    string s;
    cin >> s;
  }
  return 0;
}

When you run the program, RGB images are displayed. Exit with 'q' key.

Since calling the setBodyIndex() function with "false" argument, the obtained bodyIndex image is an image of cv::Vec3b type in each pixel. In the example execution, as the person being recognized is represented by cyan (light blue), we can see that his bodyIndex is 3.

Please click here for this sample project KinectV2_bodyIndex.zip.

Since the above zip file may not include the latest "NtKinect.h", Download the latest version from here and replace old one with it.

http://nw.tsuda.ac.jp/