Darwinbots Forum

Code center => Suggestions => Topic started by: Prsn828 on June 23, 2010, 07:33:47 AM

Title: Bot Code Analysis Tool
Post by: Prsn828 on June 23, 2010, 07:33:47 AM
First version:
http://www.mediafire.com/?i3pshzg3os73luk (http://www.mediafire.com/?i3pshzg3os73luk)

This one isn't so much a suggestion as it is me stating what I'm going to try to do.
I figured this would be a good place to get commentary on the idea.

Here's what I have in mind:

Some sort of (probably java based) text editor that can take a gene and use an algorithm to parse bot code into infix notation using parenthesese.  I'd want to put other things in too, but for starters I think that alone would be helpful.

The other thought I had was to allow it to provide toggling between integers and their associated sysvars, so we could try to decipher evolved bots' codes more easily.

I haven't worked on any coding of bots recently, but I'm very confident I can get this working (and it'd help me to write some bots too).

I'm also thinking I might be able to have it work the other way, being able to take infix and translate it into postfix (while also preserving the infix version, or parsing it back for later editing).

Does this sound like something that would be helpful, or am I being overly ambitious here?
Title: Re: Bot Code Analysis Tool
Post by: Ammeh on June 23, 2010, 08:28:11 AM
That sounds amazingly useful. If you think you can do it, go for it :D
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on June 23, 2010, 08:38:34 AM
I'm sure I can do it, but it will take me some time  :P
I'll work on it when I get home today.
Title: Re: Bot Code Analysis Tool
Post by: Numsgil on June 23, 2010, 02:37:37 PM
The hard part really would be parsing the code.  Not that it's hard conceptually just that it's a bit of a project, and everything else (conversion from postfix to infix) you can look up online somewhere.

You might also want to try something of an "optimizing compiler".  ie: given an evolved bots code you can prune out junk DNA which you know can't influence the bot's behavior.

Like if you had a bot that did:

0 0 10 sqrt 400 10 .repro store mul 0 0

You can do some static analysis and get that the "effective" DNA is only:
10 .repro store

Likewise if something does:

10 10 3 mul dup mul 9 div store

Since those are all constants you can collapse it down to:

10 300 store

which is the same as:

10 .repro store.

...

I want to add something like this to DB3 eventually (basically the end goal is a bot writing IDE that integrates into the program, let's you single step debug through bot code, watch windows, etc.), so if you make the code clean and readable we can probably steal some of it when the time comes.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on June 23, 2010, 03:55:18 PM
Actually, I was thinking of taking it even further, and letting it tell the user what genes, for instance, might rely on one another.

That comes later though, first is getting it running with basic conversions from postfix to infix and identifying where numbers are being used as sysvars, etc.
Title: Re: Bot Code Analysis Tool
Post by: Ta-183 on June 25, 2010, 05:08:48 PM
Really, I would probably kill for a tool like that. It would likely spell the difference that launches DB from an obscure a-life simulator used by only a couple dozen and upheld by even fewer to the 'killer app' of the programming game nerd. Most people that I tell about darwinbots think it's a neat idea until I reveal that the language used is a reverse-polish assembler. This could also really influence the creation of more and more complex league bots and would prove invaluable in the far more complex environment of DB3. Bots with complex diagnostic, error handling and antivirus systems would be far simpler to create and become far more common. The applications for bot development that such a program would have are boundless. The simple use of it as an error-checking and debugging tool would have far more immediate usefulness.

Who among us DOESN'T want to go back and make our best bots behave exactly as we wanted them to? It love to make SF2 work correctly. Half the time, I don't even know what the hell it's doing. Such is the reality of implementing complex structures in an esoteric assembly language. (DB4.0, now in Brainfuck!)
Title: Re: Bot Code Analysis Tool
Post by: Numsgil on June 25, 2010, 08:39:25 PM
(DB4.0, now in Brainfuck!)

Hmm...

:)
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on June 26, 2010, 11:05:30 AM
Well, I've managed to get it started, so we'll see what happens.

It won't be too hard to get the first part, the infix to/from postfix, to work correctly.
When it does I might release a beta version of the tool (more accurately I'd say it's like an alpha).

Also, I've got it structured so that all of the actions it performs are in utility classes, so it should be easy to incorporate them into other programs.

If I do this right, you will be able to code in either infix or postfix, or can switch between the two.

I'll keep everyone informed of the progress as I get parts of it working.

[EDIT:]
I have the postfix to infix functionality working.  I tested it on Jeffalish from the bestiary, and it seems to work properly.  I don't know how well it might do with evolved code though.
The next step is infix to postfix.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 17, 2010, 08:02:45 AM
I have ended up scrapping the Java in favor of Python.  As a bonus, it appears that I will now be able to add in tooltips, and have them directly pointed at the wiki (Ex: if your cursor is after the word add, and you press control + space, a small window appears with the wiki page for the add command.)

It uses WxPython, so anyone who wants to use it will also have to install Python and WxPython, but for anyone interested in something as code oriented as writing bots, this really shouldn't be a problem.

I have to re-implement the postfix/infix translations, but that shouldn't be too hard to do.

I'll release a beta once I get at least one of the useful features working fully. (Besides load, save, and saveAs, which I already have built into it)
Title: Re: Bot Code Analysis Tool
Post by: Shasta on August 18, 2010, 12:01:25 AM
When you put out a version, could I request using py2exe (http://www.py2exe.org/)? It makes way easier for everyone to just download and use it.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 19, 2010, 10:19:46 AM
When you put out a version, could I request using py2exe (http://www.py2exe.org/)? It makes way everyone to just download and use it.
Sure, that sounds like a good idea to me.
Hopefully I will get some spare time to work on it soon.  No garuntees though, as I have been pretty busy lately.
I also might have hit a snag on the tooltip idea, but I won't know for sure until I have the time to try it out.

I'll be sure to keep everyone updated whenever I make any progress.
Title: Re: Bot Code Analysis Tool
Post by: Panda on August 19, 2010, 02:11:27 PM
Please do keep us updated. This is sounding good.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 20, 2010, 10:18:07 AM
[Edit] I have this mostly figured out now.  All that is left is to find the time to code it.
          Might not be able to get to it for a while though.


After doing a little searching, I believe I have found a nice solution for distribution of the program.
While py2exe is nice and referenced quite often, I have found PyInstaller, which appears to be much simpler for both the user and the programmer (no license issues or .dll's to worry about).

Most likely I will make some decent progress this weekend.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 23, 2010, 08:34:52 AM
I have some more news for everyone.

I have all of the basic preparations complete.
IDE is set up, I have decided what libraries and extensions I will be using, and I have a good mental picture of how it will end up looking.

It is going to require that you have Internet Explorer installed at first, but I will add the option to use different browsers later.
This requirement is to facilitate code tooltips directly linked to the wiki.

I am also playing with the idea of adding in an obfuscation functionality.  I figure it could cause some pretty interesting evolution since it could separate the human aspect from the code.

postfix to and from infix will be supported, and I will likely distinguish the two notations using hilighting.

I'll set it up so that when it saves it will be converted to postfix.
I might use comments to allow it to be read back in using the same notation it had before the conversion for saving.

I can't make any garuntees, but I may also be able to add code completion.
Less likely but also still within the realm of possibilities is offline wiki support (basically storing a copy of the wiki locally).

I hope to also attempt a partial interpreter, allowing in-editor debugging. Wish me luck with this, but please don't hold your breath waiting for this feature.

I'll probably get the structure of the program figured out by the time I leave work today (gotta love spare time), so expect more news on that later today or tomorrow.
Title: Re: Bot Code Analysis Tool
Post by: Numsgil on August 23, 2010, 05:26:29 PM
I hope to also attempt a partial interpreter, allowing in-editor debugging. Wish me luck with this, but please don't hold your breath waiting for this feature.

(http://www.smileystar.com/smileys/msn-crazy-smileys/holding%20breath.gif)
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 24, 2010, 09:47:08 AM
I've worked out a mental image of the program structure.
Not that it's important from a user perspective, but I'm basically going to use seperate files (modules) for each functionality.

As far as the interpreter is concerned, I believe I have thought of a good way to handle that.
It will take a while to program though...

Related to the interpreter, I have come up with an interesting thought.
It appears to me that the transition between frames has an effect on a bot's variables that could be represented (with the exception of external stimuli) by something like an excel sheet filled with formulas.
It might be a stretch, but I think I may be able to design an interpreter FORMAT, meaning different formats could represent different versions of darwinbots, or different settings.
The same can also be applied to the sysvars, although I'm also thinking about trying to parse them from the wiki to ensure they are up to date (maybe a file override of sysvars could be allowed as well...)

If I do get a working format for the interpreter, we may want to consider integrating it into darwinbots (since we certainly aren't overwhelmed by the amount of settings it has right now  :P)
This would make tweaking the behavior and balance of the program much easier to do.

Now that I think about it, I wonder if it is possible to treat bot-code as a programming language (and as a result, I could make a higher level language that compiles into bot-code...)
I'll play around with that idea a bit after I get some of the other parts coded (or when I can't control my ADHD any more and it forces me to think about it  :P)

If anyone can point me to information about how I might query the wiki, it would be greatly appreciated.
As it is right now I can only really display the full page when attempting to provide tool-tips, and without being able to retrieve any actual (read: useful to a program) information, I can't really provide any auto-complete features without hard-coding (or locally storing) all of the related information.
Information in this case refers to: valid keywords (add, sub, mod, etc.), the list of sysvars (and their numbers), the usage of keywords (number and order of arguments), etc.

I'll stop at that for now.
As always I will keep everyone (well-)informed.
Title: Re: Bot Code Analysis Tool
Post by: Botsareus on August 24, 2010, 06:56:16 PM
Here is a thought:

Lets say we got:
Quote
*50 2 add 1 sub 50 store

Can you make the thing you are working on suggest

Quote
50 inc

Or, here is another example

Quote
*50 *50 mult 50 store

program suggests:

Quote
*50 2 pow 50 store

Please make this work on 'suggest' bases. We don't need the program changing stuff automatically bc there might be an evolutionary advantage in the long-form.
Thank you, keep up the good work.

Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on August 25, 2010, 09:07:42 AM
Here is a thought:

Lets say we got:
Quote
*50 2 add 1 sub 50 store

Can you make the thing you are working on suggest

Quote
50 inc

Or, here is another example

Quote
*50 *50 mult 50 store

program suggests:

Quote
*50 2 pow 50 store

Please make this work on 'suggest' bases. We don't need the program changing stuff automatically bc there might be an evolutionary advantage in the long-form.
Thank you, keep up the good work.

It will be as you have suggested.
All of those are actually quite good examples of what I am eventually hoping to make it capable of.

In addition, I will probably add in "pointer inference", or the ability to determine if a value (or expression) is supposed to evaluate to a memory location reference.
For individual values followed by the reference operator, this is trivial.
For more complex expressions it could be useful in decyphering the meaning of an evolved code.

I also might add evolution-based optimization.

To explain:

Consider a line of code in a bot.
This code could be considered like a function, as it is deterministic in behavior (except when it involves rnd, but that doesn't really change anything).
By applying a fitness and evaluating mutations of the function, one could evolve a new function, having the same effect on the memory of the bot, but using more or less expressions (or expressions with more or less energy requirements).
Of course, given the resulting change to the memory of the bot, it is theoretically even possible to evolve the code from nothing, but I don't know how useful such a feature might be. (a preson can generally write the code faster than the changes they want it to make, I believe).

And as you said, it would only be suggestions.
It would be quite rude to change the code of the bot without asking  :P
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 01, 2010, 09:27:19 AM
I have a working browser in place, so I can now garuntee wiki-based command descriptions. (Unless the wiki gets removed  :P)

I switched to Qt as it has a GUI development tool, and is more fully documented than WxPython.  It also has a better browser in its toolkit.

I have the input capturing of the editor set up too, and am going to move handling of the editor's special functionalities into a seperate file soon (right now it is laying around in the initializer).

Thanks to the way Qt's text widget works, I should be able to save in-progress bots along with some meta-data in an HTML format.  This is the format the widget uses for formatting, and it has built-in support for reporting as either fully formatted HTML or as plain text.  This way I could keep track of what sections are post-fix and what sections are in-fix without any extra coding.

As far as an interpreter goes, I'm considering various approaches, but have not settled on anything yet.

Code conversion will be divided into three categories (or at least right now they are):
All three may overlap a little, but for the most part they are distinct.

Mutation is just that, the mutation of the code, whether by switching out for equivalent statements or changing the code altogether.

Evolution makes use of mutation to derive more (or less, if you really want it to) efficient, effective, or desirable (whatever you choose for it to mean) implementations of a section of code.
I might also allow evolving code to meet certain specifications somehow, but for now it's just a thought.

Conversion is more concrete.
It deals with post-fix to/from in-fix, as well as any other non-random algorithm-based conversions (like sysvar to numbers, and comment/uncomment of code)

As always, I will tell you more when I get some more work done on this.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 05, 2010, 11:16:09 AM
Big update!

I have the wiki searching working (it definitely isn't in its final state, but it is perfectly usable).
I also have the menu set up.

Right now I am working on implementing the load and save mechanisms (prompts, formats, and all that good stuff).

As soon as saving and loading is complete I will make an executable and provide a link to a download for it.

As always, more news when it happens.
Title: Re: Bot Code Analysis Tool
Post by: Ammeh on September 06, 2010, 03:19:29 AM
Awesome, looking forward to this  :happy:
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 08, 2010, 08:58:39 AM
I've taken some time off of work (Five day weekend  :P ), so I will probably be able to get a release out sometime between Friday morning and Tuesday night.

I have loading working already, so now I just need the saving part, which for now will be plain and simple.
Hopefully I can make the excecutable without too much difficulty.

On a slightly related note, I have realized that using the search functionality (that searches the wiki when you press ctrl+space) works not only on words, but also on numbers.  This will surely come in handy when I try to work out some of the conversion functionality.  Of course, that also means we will need to take slightly better care of our wiki (It's mostly fine right now, but some pages don't seem to follow the conventions that the rest of the wiki uses).

More news to come soon (most likely once I have an excecutable).
Title: Re: Bot Code Analysis Tool
Post by: Shasta on September 09, 2010, 08:26:29 PM
The wiki could really use some maintenance. I started to tidy some things up... and got busy with other stuff. I'll hopefully get around to it in a little while but if someone has nothing to do help would be great :)
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 11, 2010, 08:29:32 AM
Saving and loading are working now!

Maybe an hour before I release the first (sub-alpha) version!

There is lots more work to be done on it, so don't expect this to be all that great (although it is nice to have the power of the wiki at your disposal.)
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 11, 2010, 09:49:24 AM
Link to the download is on the first post!

Eventually I will change the name of it, but this is a very quick release, so for now just bear with me.

On another note, now that I have the first release out, I will be adding features.

Any suggestions are more than welcome.
Title: Re: Bot Code Analysis Tool
Post by: Ammeh on September 11, 2010, 03:58:09 PM
Downloading now...

12.3Mb O_o

Before you do a final release, get shasta or nums to give you a link the exe compressor thing. It's pretty damn effective.

Edit: maybe I'm doing something wrong, but all I can do is open, save or save as. When I open it loads it into a textbox, but that's all.
Title: Re: Bot Code Analysis Tool
Post by: Billy on September 11, 2010, 04:52:15 PM
I'm getting the same problem as Sammeh, it doesn't do anything. How do you get it to work?
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 11, 2010, 05:35:52 PM
Press control-space to run a search on the wiki.  Right now load, save, and the search, are the only features I have so far, but now that it is at a usable point it will be easier to add new features to it.
Title: Re: Bot Code Analysis Tool
Post by: Shasta on September 11, 2010, 05:51:56 PM
12.3Mb O_o

Before you do a final release, get shasta or nums to give you a link the exe compressor thing. It's pretty damn effective.
I just tried to compress it. Got an error saying it had already been packed by upx. :blink:
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 11, 2010, 06:24:00 PM
I'm not surprised. It's written in python, so it has a bootstrapped version of python inside of it, besides all of the code for the actual program.  Of course, this also means that it won't grow in size very much as new features get added  :P
Title: Re: Bot Code Analysis Tool
Post by: Ammeh on September 11, 2010, 06:26:59 PM
*reaffirmed beliefs that python is evil*

>.>

If you wanna post the code after you're done doing all the hard bits :P I can rewrite it in Delphi or something. exe will be a fraction of the size.
Title: Re: Bot Code Analysis Tool
Post by: Shasta on September 11, 2010, 07:51:35 PM
12MB really isn't that bad, unless you are on dialup. I wouldn't worry about it.
Title: Re: Bot Code Analysis Tool
Post by: Billy on September 11, 2010, 07:56:23 PM
I can download it in less than a minute.

Okay, I can use it now. This should be quite useful. Can't wait for postfix/infix toggling.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 13, 2010, 08:00:33 AM
Postfix/Infix should be the next addition, following when I figure out how I want to format the file so that it can be saved with or without (depending on user selection) infix code remaining in the save file.

It'll probably end up having some HTML style tags in the code when infix is saved into a file, so that it can remove the tags and translate back when it is saved for use by Darwinbots.
Title: Re: Bot Code Analysis Tool
Post by: Prsn828 on September 17, 2010, 08:19:07 AM
I've been doing a little work on thinking about the interpreter.

Depending on how I decide to do it, it may come before postfix/infix translations.

Speaking of postfix/infix, I need some opinions on this.
Should I go all-out parenthasizing to the point where it looks like Lisp turned into Darwinbots infix, or should I use order of operations? (PEMDAS) (Parentheses-Exponent-Multiplication-Division-Addition-Subtraction for those who might be drawing a blank here).

I really need a good idea of what people want (unless you want it to be an option).

It might even be worth thinking about using grammars or something similar to automate the translations, and to make them customizable.

By the way, I do have a way in mind to avoid the problems of unknowns when writing the interpreter.
It's kind of like an algebra system, where unknowns are left in the system as values, allowing the evaluation to progress to any number of steps.


For instance, when evaluating:
Code: [Select]
*.eye5 2 addWe get the result:
Code: [Select]
(*.eye5 + 2)Which would then be on the stack, and is treated as if it were just another integer value.

The hardest part will be dealing with code branching, but at first I can simply use the trinary a?b:c operator whenever one of those comes up (it'll be pretty ugly though).

One issue with this is that *.eye5 could (and often should) have different values for different time-steps.
The solution is to make an object to replace the string *.eye5, and have that object track which step it is from.

Then, after the code has been interpreted this way for some number of steps, the user could input the unknowns to debug their robot.

While tedious, I do believe this will be a very powerful and helpful tool when completed.

Give me some feedback so I can get a better idea of what I am aiming for here.
Title: Re: Bot Code Analysis Tool
Post by: Botsareus on February 09, 2011, 11:15:00 AM
How about this:

One text box with the original code.
One with the 'simplified' code that is still in db.

When you select a portion of the simplified code it displays a 3rd text box with the 'real order of operations' for that selection.

If you modify the third textbox it changes the selected code in the second textbox.

P.S.

Prsn , keep us updated...