NtKinect: Kinect V2 C++ Programming with OpenCV on Windows10

How to get BodyIndex image with Kinect V2


2016.07.20: created by
2016.07.21: revised by
Japanese English
To Table of Contents

Prerequisite knowledge


BodyIndex image

BodyIndex images can be acquired with a resolution of 512 x 424 as well as Depth images. Up to six people can be distinguished at the same time.

In NtKinect, the obtained BodyIndex image is represented by uchar or cv::Vector3b for each pixel.

NtKinect's functions for BodyIndex image

type of return valueċž‹ function name descriptions
void setBodyIndex(bool raw = true) Set BodyIndex image to the member variable "bodyIndexImage"
If the argument raw is "true" or there is no argument, the type of each pixel is uchar.
If the argument raw is "false", the type of each pixel is cv::Vec3b.

NtKinect's member variable for BodyIndex image

type variable name descriptions
cv::Mat bodyIndexImage BodyIndex image.
Up to six people can be detected at the same time, and BodyIndex itself is a number from 0 to 5 assigned to each detected person. The type of each pixel of the BodyIndex image may be uchar or cv::Vec3b. The coordinates of the image are positions in the DepthSpace coordinate system.
  • bodyIndex.cols --- Resolution for the horizontal direction of the image (= 512)
  • bodyIndex.rows --- Resolution for the vertical direction of the image (= 424)
  • bodyIndex.channels() --- Return 1 if the pixel is of type uchar, and 3 if cv::Vec3b.

In case of uchar data type

The value of each pixel is 0 to 5 in the pixel where a person is detected, and 255 in the other pixels.
  • bodyIndexImage.at<uchar>(y , x ) --- Access pixel at the (x , y ) coordinates of the image
  •     uchar pixel = bodyIndexImage.at<uchar>(y , x )
    

In case of cv::Vec3b data type

The value of each pixel is the RGB value corresponding to the bodyIndex number.
  • bodyIndexImage.at<cv::Vec3b>(y , x ) --- Access pixel in the (x , y ) coordinates of the image
  •     cv::Mat pixel = bodyIndexImage.at<cv::Vec3b>(y , x )
    
    bodyIndexcv::Vec3bcolor
    025500  
    102550  
    200255  
    32552550  
    42550255  
    50255255  

How to write program

  1. Start using the Visual Studio's project KinectV2.zip of "NtKinect: How to get RGB camera image with Kinect V2 (Fundamental Settings)"
  2. Change the contents of "main.cpp" as follows.
  3. Call kinect.setBodyIndex() function to set bodyIndex data for each pixel in kinect.bodyIndexImage.

    main.cpp
    #include <iostream>
    #include <sstream>
    
    #include "NtKinect.h"
    
    using namespace std;
    
    void doJob() {
      NtKinect kinect;
      while (1) {
        kinect.setBodyIndex(false);
        cv::imshow("bodyIndex", kinect.bodyIndexImage);
        auto key = cv::waitKey(1);
        if (key == 'q') break;
      }
      cv::destroyAllWindows();
    }
    
    int main(int argc, char** argv) {
      try {
        doJob();
      } catch (exception &ex) {
        cout << ex.what() << endl;
        string s;
        cin >> s;
      }
      return 0;
    }
    
    
  4. When you run the program, RGB images are displayed. Exit with 'q' key.
  5. Since calling the setBodyIndex() function with "false" argument, the obtained bodyIndex image is an image of cv::Vec3b type in each pixel. In the example execution, as the person being recognized is represented by cyan (light blue), we can see that his bodyIndex is 3.




  6. Please click here for this sample project KinectV2_bodyIndex.zip.
  7. Since the above zip file may not include the latest "NtKinect.h", Download the latest version from here and replace old one with it.



http://nw.tsuda.ac.jp/