Controll ar_piano Sinbad with human body and kinect tracker?

yaozr · 3-Jun-2011 17:41:56

This is the actual wanted kinect AR in my view.

In other word, I'm interested in using NITE output as the "armarker" instead of the paper marker, I found two places of the relevant plugins, one is svn/plugins/BitmapToolkit and the other is svn/plugins/OpenNISCOL, my understanding is that I have to create one plugin of my own, say, a ArKinectMarker, and it'll take NITE skeleton as the "marker" and can attach one object accordingly as what "AR marker Sinbad" does in the example, this behavior is not exactly the same as the "object follow" example used in kinect skeleton demo, since I also want the rendered object to match with the input 2d video from Kinect RGB camera, instead of just imitating the subject body's motion in 3d space, as my primary goal is not gaming but AR.

It seems to me that all the elements are there and I need somehow package it as a new plugin into the editor, but I failed to find document on how to customize or add a new plugin to the existing system.

Any shedding of light would be greatly appreciated!

Yaozr

arkeon · 3-Jun-2011 22:06:06

Sorry for the lack of plugIT API documentation ^^ this is a part of my todo list since some months
For a kinect user plugIT with bone positions matching with the camera picture (the kinect device plugIT already have all what you need including camera projection matrix when AR mode is checked)
I think that we just need to transform all joint position in the 3d camera space.

To make plugIts you must start to be familiar with Scol language, you can start here http://redmine.scolring.org/projects/sc … olLanguage and with the tutorials available on iri"s website.

yaozr · 4-Jun-2011 12:53:08

Many thanks Arkeon for your prompt response! still trying to digest your hint...

Do you mean in Kinect device plugIT it already get all bone positions and this should be enough to drive the Kinect user plugIT directly without an extra camera projection, the effect should be as the original NITE skeleton sample and we can pose sinbad according to bone's 2d position? in that case the sinbad is scalling at each frame to fit the screen image.

Or do you mean after Kinect device plugIT we can add one step of transform joint positions in 3d camera space so that the projection is one front view projection (or orthogonal projection?) and we get a realistic 3d body of sinbad at same size as human subject, in that case the sinbad is keeping the same physical size as human and not scale each frame.

BTW I did my first trail of a clean building of the OpenniSCOL vs2008 project, and failed at starting point of error msg such as in ScolPlugin.cpp "cbmachine ww;" the error says that cbmachine was not defined. I was confused about the project setting since it seems the sdk/include/SCOL/scolplugin.h was never included so the error msg seems make sense, the project include only the other ScolPlugin.h file in project directory, as I did not find the documentation about sdk path I let system environment variable "SCOL_SDK" point to "SVN\scol_sdk\include\SCOL" I hope that is correct? Tips are greatly appreciated.

BR/Yaozr

Last edited by yaozr (4-Jun-2011 13:02:45)

arkeon · 4-Jun-2011 13:45:18

more this one

Or do you mean after Kinect device plugIT we can add one step of transform joint positions in 3d camera space so that the projection is one front view projection (or orthogonal projection?) and we get a realistic 3d body of sinbad at same size as human subject, in that case the sinbad is keeping the same physical size as human and not scale each frame.

for the scol_sdk, there is a setup in svn\trunk\tools\plugin sdk\release

yaozr · 4-Jun-2011 15:30:41

thank you arkeon!

yaozr

Scolring - Forum

#1 3-Jun-2011 17:41:56

Controll ar_piano Sinbad with human body and kinect tracker?

#2 3-Jun-2011 22:06:06

Re: Controll ar_piano Sinbad with human body and kinect tracker?

#3 4-Jun-2011 12:53:08

Re: Controll ar_piano Sinbad with human body and kinect tracker?

#4 4-Jun-2011 13:45:18

Re: Controll ar_piano Sinbad with human body and kinect tracker?

#5 4-Jun-2011 15:30:41

Re: Controll ar_piano Sinbad with human body and kinect tracker?

Board footer