Board Logo
« simulated mouse clicks. typing and image recogniti »

Welcome Guest. Please Login or Register.
Jan 18th, 2018, 3:49pm


Conforums Terms of Service | Membership Rules | Home | Search | Recent Posts | Notification | Format Your Message | Installation FAQ


« Previous Topic | Next Topic »
Pages: 1  Notify Send Topic Print
 thread  Author  Topic: simulated mouse clicks. typing and image recogniti  (Read 814 times)
mmiscool
Member in Training
ImageImage


member is offline

Avatar



AIM
Homepage PM

Gender: Male
Posts: 62
xx simulated mouse clicks. typing and image recogniti
« Thread started on: Aug 3rd, 2014, 11:17pm »

Hello,

I have been trying to find a free and easy to use macro recorder for some time.
Most of the ones on the market are some what dificult to use and are costly.

I wanted some thing I could interface with LB for the programing with the following features.

Image recognition for detecting if some thing is on the screen.

automatically clicking the mouse using image recognition snapshot of what is to be clicked.

For this I created a stand alone command line program that receives specific command line arguments and then exicutes the proper action. I built it with autoit and it is quite capable of being used in bat files or from the LB run command.


Here it is

https://sites.google.com/a/smbisoft.com/www/fun-projects-don-t-look/file?pli=1



Download Doit.zip for the package. It is about half a meg. I tested on window 7 32 bit but it should work on vista and xp. Have not tested with windows 8 however.

I uploaded the exe file and source code here for the command line program. It was written in Autoit and compiled with the console property to enable standard out.



In liberty basic it can be used by the run command.

some thing to note is that it is case sensitive so all parts of the command must be in lower case except the image name or the text to be type.
Code:
Run "doit click left image.bmp 0 0"
 




I included a help file with the program.

Will return 1 for success and 0 for failure.

From command line use the following


Singe click
doit click {right/left} {image.bmp/0 (for no image)} {x offset} {y offset}

example: Doit click left image.bmp 5 8
without image and absolute quadrants
doit click left 0 150 250


Double click (Same as click except it double clicks)
doit dclick {right/left} {image.bmp/0 (for no image)} {x offset} {y offset}




Simulated keyboard typing
Doit type "string to be typed"


Detect if an image is on the screen
doit detect {image.bmp}

User IP Logged

Check out the code wiki at http://smbisoft.com
The code wiki allows for multiple users to work on the same project at the same time in just basic.
JustBasic 1.2, LibertyBasic 4.4, Ubuntu 10.10, AMD Ultimate vision, 6 core, 3 ghz, 8 gig ram.
SMBISoft ____888-LAN-Zoli _____ 888-526-9654
Mallot1
Full Member
ImageImageImageImage


member is offline

Avatar

Mallot1


Homepage PM

Gender: Male
Posts: 194
xx Re: simulated mouse clicks. typing and image recog
« Reply #1 on: Aug 4th, 2014, 2:28pm »

Very useful! Great job! cheesy -Mallot1
GameGrape Corporation: www.gamegrapecorporation.blogspot.com
User IP Logged

User Image
User Image
Game Development Blog: http://mallot1development.wordpress.com
High Flyer! cheesy http://gamegrapestudios.com/highflyer
Rod
Administrator
ImageImageImageImageImage


member is offline

Avatar

Graphics = Goosebumps!


PM

Gender: Male
Posts: 3151
xx Re: simulated mouse clicks. typing and image recog
« Reply #2 on: Aug 4th, 2014, 3:07pm »

Actually it is so amazing I recon folks are taken aback. First of it is quite a stretch to imagine how you might implement the code. Not saying it is difficult just that it is so different it requires quite a lot of thinking.

Basic concept is that we can script any button press or any text input on any screen!

First thought, not being picky but transparency, how do we handle transparency which makes many buttons, and icons different on different user screens and how do we discern the difference between five cancel/close X buttons on the same desktop.

Anyways very interesting concept.
« Last Edit: Aug 5th, 2014, 03:04am by Rod » User IP Logged

mmiscool
Member in Training
ImageImage


member is offline

Avatar



AIM
Homepage PM

Gender: Male
Posts: 62
xx Re: simulated mouse clicks. typing and image recog
« Reply #3 on: Aug 4th, 2014, 7:13pm »

Personally if I were to attempt to automate tasks on different machines I would make the windows theme the same on all the machines.

The nice thing though is that once you have a task automated like clicking links on a web site if they change the look and feel of the site all you have to do is update the images.
User IP Logged

Check out the code wiki at http://smbisoft.com
The code wiki allows for multiple users to work on the same project at the same time in just basic.
JustBasic 1.2, LibertyBasic 4.4, Ubuntu 10.10, AMD Ultimate vision, 6 core, 3 ghz, 8 gig ram.
SMBISoft ____888-LAN-Zoli _____ 888-526-9654
Pages: 1  Notify Send Topic Print
« Previous Topic | Next Topic »

Conforums Terms of Service | Membership Rules | Home | Search | Recent Posts | Notification | Format Your Message | Installation FAQ

Donate $6.99 for 50,000 Ad-Free Pageviews!

| |

This forum powered for FREE by Conforums ©
Sign up for your own Free Message Board today!
Terms of Service | Privacy Policy | Conforums Support | Parental Controls