Quantcast

[OpenNI-dev] Custom Gesture Recognition using OpenNI and Unity3D

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Custom Gesture Recognition using OpenNI and Unity3D

MichaelK
Hi guys,

I have uploaded a video of my first implementation for custom
gestures. It works really great - just have a look here:
http://www.youtube.com/watch?v=t89TUfjFuGs
Right now it's a beta version, but I look forward to finish the work
this month :)

Regards,
Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

kenshen
Awesome can't wait :)

On Jun 24, 8:57 am, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a video of my first implementation for custom
> gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> Right now it's a beta version, but I look forward to finish the work
> this month :)
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Kaushal
some really interesting stuff!!

On 24 June 2011 20:45, kenshen <[hidden email]> wrote:
Awesome can't wait :)

On Jun 24, 8:57 am, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a video of my first implementation for custom
> gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> Right now it's a beta version, but I look forward to finish the work
> this month :)
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.


--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Bill
In reply to this post by MichaelK
This looks excellent. Do you plan on releasing it to the community?

Bill


On Jun 24, 3:57 pm, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a video of my first implementation for custom
> gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> Right now it's a beta version, but I look forward to finish the work
> this month :)
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

MichaelK
Right now it's in beta stage. I have to discuss this with the members
of the company I am working for. Maybe I can release a dll-File which
can be used ;)

On Jun 25, 2:50 pm, Bill <[hidden email]> wrote:

> This looks excellent. Do you plan on releasing it to the community?
>
> Bill
>
> On Jun 24, 3:57 pm, MichaelK <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hi guys,
>
> > I have uploaded a video of my first implementation for custom
> > gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> > Right now it's a beta version, but I look forward to finish the work
> > this month :)
>
> > Regards,
> > Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

hassan abu al-haija
In reply to this post by MichaelK
Really awesome work !:)
I just wanna ask you if you can give us some details about the method
you are using here.
Are you using some machine learning method for learning the pose and
then classify it ? or you are just using other classical way with
saving angels and tolerances ?
did you use any other libraries than OpenNI ?  did you use the C#
wrapper of NITE ?

Sorry for too much questions, I am just too interested in it :)

cheers and congratulations for the great job.

On Jun 24, 2:57 pm, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a video of my first implementation for custom
> gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> Right now it's a beta version, but I look forward to finish the work
> this month :)
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

kenshen
yeah srs i am working on a kinect rpg and would love this it would make making animations so much easier 

On Sat, Jun 25, 2011 at 6:59 PM, hassan abu al-haija <[hidden email]> wrote:
Really awesome work !:)
I just wanna ask you if you can give us some details about the method
you are using here.
Are you using some machine learning method for learning the pose and
then classify it ? or you are just using other classical way with
saving angels and tolerances ?
did you use any other libraries than OpenNI ?  did you use the C#
wrapper of NITE ?

Sorry for too much questions, I am just too interested in it :)

cheers and congratulations for the great job.

On Jun 24, 2:57 pm, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a video of my first implementation for custom
> gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> Right now it's a beta version, but I look forward to finish the work
> this month :)
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.




--
I don't usually write emails but when i do i forget punctuation 

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

MichaelK
In reply to this post by hassan abu al-haija
I am only using OpeNI's C# wrapper and some Unity3D functions. I am
not using machine learning, but like you wrote, I use the joint-
informations together with a tolerance. I think that works better than
machine learned stuff and is much faster :)

On Jun 26, 2:59 am, hassan abu al-haija <[hidden email]> wrote:

> Really awesome work !:)
> I just wanna ask you if you can give us some details about the method
> you are using here.
> Are you using some machine learning method for learning the pose and
> then classify it ? or you are just using other classical way with
> saving angels and tolerances ?
> did you use any other libraries than OpenNI ?  did you use the C#
> wrapper of NITE ?
>
> Sorry for too much questions, I am just too interested in it :)
>
> cheers and congratulations for the great job.
>
> On Jun 24, 2:57 pm, MichaelK <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hi guys,
>
> > I have uploaded a video of my first implementation for custom
> > gestures. It works really great - just have a look here:http://www.youtube.com/watch?v=t89TUfjFuGs
> > Right now it's a beta version, but I look forward to finish the work
> > this month :)
>
> > Regards,
> > Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Naëm Baron
It's pretty neat !
The plugin seems well done with lots of features...

So you did that at work, right ? It would be cool to release it. Just
to use custom gesture and also because I am curious to see how to code
"full-integrated" unity plugin.

NB

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Sam Muscroft
Looks like a nice tool for triggering events based on static poses or
for capturing mocap data. I would have to disagree with you about
machine learning though Michael, you've made a sweeping generalisation
about a term which encompasses many different techniques and
technologies. I have successfully used svm for hand pose recognition
and it can be trained in seconds and can recognise in real-time very
accurately.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Paulo Trigueiros
Hi Sam,

I saw your message I would like to use SVM for gesture recognition.
I'm using openframeworks+code::blocks for the development. Is there any
library for openframeworks. How did you implemented it? Can you share
code, experience?
Thanks in advance,
Paulo Trigueiros

-----Original Message-----
From: Sam Muscroft <[hidden email]>
Reply-to: [hidden email]
To: OpenNI <[hidden email]>
Subject: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and
Unity3D
Date: Mon, 27 Jun 2011 03:08:04 -0700 (PDT)

Looks like a nice tool for triggering events based on static poses or
for capturing mocap data. I would have to disagree with you about
machine learning though Michael, you've made a sweeping generalisation
about a term which encompasses many different techniques and
technologies. I have successfully used svm for hand pose recognition
and it can be trained in seconds and can recognise in real-time very
accurately.


--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Jeff Winters
In reply to this post by MichaelK
Wow! This is super -- well done, MichaelK! Add me to the list of those clamouring for a source release, or failing that, a detailed description of how you used the joint information to do gesture tracking.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To view this discussion on the web visit https://groups.google.com/d/msg/openni-dev/-/ITgxc4wRgkMJ.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Jeff Winters
In reply to this post by MichaelK
Wow! This is superb -- well done, MichaelK! Add me to the list of those clamouring for a source release, or failing that, a detailed description of how you used the joint information to do gesture tracking.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To view this discussion on the web visit https://groups.google.com/d/msg/openni-dev/-/wHPJppTTgX0J.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Sam Muscroft
In reply to this post by Paulo Trigueiros
Hi Paulo,

I used pure OpenCV for the image processing and machine learning . I
am unable to share the source, but the basic principle was to track
the contours and then train the svm with contour features. This has
it's limitations as it's based on 2d descriptors...i am interested in
looking at using point clouds with the kinect to gain better 3d shape
descriptors, there is already some work done on this by ROS (http://
pointclouds.org/).

If you're interested in machine learning with computer vision, I
recommend buying the OpenCV O'Reilly book(http://www.amazon.com/
Learning-OpenCV-Computer-Vision-Library/dp/0596516134), the machine
learning chapter is very well written and explains the core concepts
well at a high level.

An important thing to remember is that the features with which you
train your classifier are unique, distinct and repeatable...garbage in
= garbage out. With this in mind it is possible to classify most types
of data. I've found svm to perform very well on smaller data sets and
there is no lengthy training period as there is with some decision
tree models.

If you're at all interested in this, it is well worth reading around
the subject rather than learning by modifying code, you'll gain a
deeper understanding that way.

Sam.





>
> I saw your message I would like to use SVM for gesture recognition.
> I'm using openframeworks+code::blocks for the development. Is there any
> library for openframeworks. How did you implemented it? Can you share
> code, experience?
> Thanks in advance,
> Paulo Trigueiros
>
>
>
>
>
>
>
> -----Original Message-----
> From: Sam Muscroft <[hidden email]>
>
> Reply-to: [hidden email]
> To: OpenNI <[hidden email]>
> Subject: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and
> Unity3D
> Date: Mon, 27 Jun 2011 03:08:04 -0700 (PDT)
>
> Looks like a nice tool for triggering events based on static poses or
> for capturing mocap data. I would have to disagree with you about
> machine learning though Michael, you've made a sweeping generalisation
> about a term which encompasses many different techniques and
> technologies. I have successfully used svm for hand pose recognition
> and it can be trained in seconds and can recognise in real-time very
> accurately.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Paulo Trigueiros
Hi Sam,

Thanks for your answer. I going to try it and i will give some feedback.
I'm already using OpenCV inside openFrameworks for other things, so I
will look at the machine learning stuff.
Thanks again,
Paulo

-----Original Message-----
From: Sam Muscroft <[hidden email]>
Reply-to: [hidden email]
To: OpenNI <[hidden email]>
Subject: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and
Unity3D
Date: Tue, 28 Jun 2011 10:02:08 -0700 (PDT)

Hi Paulo,

I used pure OpenCV for the image processing and machine learning . I
am unable to share the source, but the basic principle was to track
the contours and then train the svm with contour features. This has
it's limitations as it's based on 2d descriptors...i am interested in
looking at using point clouds with the kinect to gain better 3d shape
descriptors, there is already some work done on this by ROS (http://
pointclouds.org/).

If you're interested in machine learning with computer vision, I
recommend buying the OpenCV O'Reilly book(http://www.amazon.com/
Learning-OpenCV-Computer-Vision-Library/dp/0596516134), the machine
learning chapter is very well written and explains the core concepts
well at a high level.

An important thing to remember is that the features with which you
train your classifier are unique, distinct and repeatable...garbage in
= garbage out. With this in mind it is possible to classify most types
of data. I've found svm to perform very well on smaller data sets and
there is no lengthy training period as there is with some decision
tree models.

If you're at all interested in this, it is well worth reading around
the subject rather than learning by modifying code, you'll gain a
deeper understanding that way.

Sam.





>
> I saw your message I would like to use SVM for gesture recognition.
> I'm using openframeworks+code::blocks for the development. Is there any
> library for openframeworks. How did you implemented it? Can you share
> code, experience?
> Thanks in advance,
> Paulo Trigueiros
>
>
>
>
>
>
>
> -----Original Message-----
> From: Sam Muscroft <[hidden email]>
>
> Reply-to: [hidden email]
> To: OpenNI <[hidden email]>
> Subject: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and
> Unity3D
> Date: Mon, 27 Jun 2011 03:08:04 -0700 (PDT)
>
> Looks like a nice tool for triggering events based on static poses or
> for capturing mocap data. I would have to disagree with you about
> machine learning though Michael, you've made a sweeping generalisation
> about a term which encompasses many different techniques and
> technologies. I have successfully used svm for hand pose recognition
> and it can be trained in seconds and can recognise in real-time very
> accurately.


--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

rusu
Administrator
In reply to this post by Sam Muscroft
Sam,

On 06/28/2011 10:02 AM, Sam Muscroft wrote:
> Hi Paulo,
>
> I used pure OpenCV for the image processing and machine learning . I
> am unable to share the source, but the basic principle was to track
> the contours and then train the svm with contour features. This has
> it's limitations as it's based on 2d descriptors...i am interested in
> looking at using point clouds with the kinect to gain better 3d shape
> descriptors, there is already some work done on this by ROS (http://
> pointclouds.org/).

Just as a clarification: the PCL project is standalone and has nothing to do with ROS. ;) Most of its developers are
geographically distributed around the world, with the OpenCV team working on PCL as well (see
www.pointclouds.org/about.html).

Cheers,
Radu.
--
Point Cloud Library (PCL) - http://pointclouds.org

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Sam Muscroft
Hi Radu,

Apologies, I thought there was some affiliation between ros and pointclouds.org.There's a link to documentation about pcl on the ros.org website (http://www.ros.org/wiki/pcl), hence I added 2+2 and came up with 5. Again, Sorry for any confusion caused.

Cheers,

Sam.

On Wed, Jun 29, 2011 at 5:41 PM, Radu B. Rusu <[hidden email]> wrote:
Sam,


On 06/28/2011 10:02 AM, Sam Muscroft wrote:
Hi Paulo,

I used pure OpenCV for the image processing and machine learning . I
am unable to share the source, but the basic principle was to track
the contours and then train the svm with contour features. This has
it's limitations as it's based on 2d descriptors...i am interested in
looking at using point clouds with the kinect to gain better 3d shape
descriptors, there is already some work done on this by ROS (http://
pointclouds.org/).

Just as a clarification: the PCL project is standalone and has nothing to do with ROS. ;) Most of its developers are geographically distributed around the world, with the OpenCV team working on PCL as well (see www.pointclouds.org/about.html).

Cheers,
Radu.
--
Point Cloud Library (PCL) - http://pointclouds.org

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

MichaelK
In reply to this post by MichaelK
Hi guys,

I have uploaded a second video of my custom gesture recognition:
http://www.youtube.com/watch?v=dQz78rWVBwA

Regards,
Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Sam Muscroft
Nice work Michael ... looks like a very useful tool. 

A quick question. Are the gestures always based on absolute co-ordinates. I mean where you've recorded the swipe gesture with your arms above your head, do you always have to have your hands in the same start position in order to begin the recognition of that particular gesture? 

Last time I was looking into motion based gestures, I was using openCV and looking at motion history gradients where a silhouette of the object being tracked (a hand in my case) was captured over time and the difference in angle of movement captured for each frame / silhouette. I suspect it would be possible to use this technique along with your approach so that you could store arrays of the skeleton point co-ordinates over time and track the deltas of the positions rather than the positions themselves so that you could detect movement in a relative fashion to make it more flexible. Just an idea...









On Tue, Jul 12, 2011 at 5:24 PM, MichaelK <[hidden email]> wrote:
Hi guys,

I have uploaded a second video of my custom gesture recognition:
http://www.youtube.com/watch?v=dQz78rWVBwA

Regards,
Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.


--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[OpenNI-dev] Re: Custom Gesture Recognition using OpenNI and Unity3D

Jman
In reply to this post by MichaelK
Hey Michael,

Your work on this is incredible.  I'm working on a similar project
in .NET and you have given me a lot of inspiration to move ahead.
Keep the videos coming!

- Justin

On Jul 12, 12:24 pm, MichaelK <[hidden email]> wrote:
> Hi guys,
>
> I have uploaded a second video of my custom gesture recognition:http://www.youtube.com/watch?v=dQz78rWVBwA
>
> Regards,
> Michael

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

12
Loading...