Image Analysis and Mouse Control

So far , weve been able to get the image to the system , now we have got to detect the position of the Pointer and feed it as cursor position. For this we use the library calls available in windows via labview.

The mouse can only be fed with points and so the image will have to be analysed for that accordingly. This first step towards that is to threshold the image. If you connect the image out of the snap vi you may or may not be able to get your vi running as the algorithms work on a particular type of image datatypes and your camera may not be providing images in that format. In order to be able to over come this the images will have to be typecast into the acceptable types. Use the image typecast vi for this.

After this works , use the threshold VIs and check the outputs on the screen , when satisfactory move on.
Since the thresholding done here is very basic , the autothreshold vi can be used , saving time and manual effort in setting up the threshold.

The threshold image output is then fed to the Centroid VI which finds out the centre of the BLOB.
This output can be fed to the "setcursorpos" library function which will set the mouse on the screen at that location.

If the mouse if to be independent and another pen is to be moved , the draw vi's can be used , which can be draw point , draw line , etc...

The block diagram is given for reference with the location of where these VI's are located in the palette.

Vision and Motion >> Vision Utilities >> Image Management >> Cast Image
Vision and Motion >> Image Processing >> Processing >> AutoBThreshold 2
Vision and Motion >> Image Processing >> Analysis >> Centroid
Programming >> Graphics and Sound >> Picture Functions >> Draw Point

Create constants, controls and indicators wherever required so as to run the VI successfully.

Block Diagram

Front Panel

After sufficiently exploring what else can be done , it may look something like this :

Block Diagram

Front Panel

The Above block shows how both the pen and the mouse can be controlled via the VI in runtime.
Also notice that while the previous block diagrams used the "snap" vi , this one uses the grab vi. This is to capture frames continuously unlike the snap function wher eit only gets one frame at a time , hence depending upon the process the intervals between each snap varies. However , in the case of the grab function , the frames are captured at regular intervals and are processed bringing about a more continuous and smooth flow of points on the screen , instead of abrupt jerks.
Now if you want to go a step ahead , try designing a VI to calibrate the camera such that it only detects area within the bounds of the screen and nowhere else , try using the region of interest tools , select region tools etc...
If I have the time I'll try uploading that as well...
Anyways... thats it for now...