Darwinbots Forum

Code center => Darwinbots3 => Topic started by: Numsgil on May 13, 2014, 04:24:05 PM

Title: Programming task (DNA editor)
Post by: Numsgil on May 13, 2014, 04:24:05 PM
I have a more-or-less stand alone project, if any coders are interested.

I'm starting to work on the GUI side of DB3.  I'd like a more-or-less full featured code editor for DNA.  There's a pretty slick code editor around that we can probably leverage: ScintillaNET (https://scintillanet.codeplex.com/).

The task is to take that as a starting point and create an IDE for DNA.  Something that I could integrate in to DB3 itself, but also run as a stand alone program.

If anyone's interested in tackling this as a summer programming project let me know and I'll help get you started.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on May 13, 2014, 04:39:12 PM
Don't know If I'll have time to help with all the other stuff going on. But some comments:

Open source code editor = cool

Quote
Something that I could integrate in to DB3 itself, but also run as a stand alone program.

That means it will have to be written in C#. And be completely object oriented.
Title: Re: Programming task (DNA editor)
Post by: Panda on May 17, 2014, 08:13:57 AM
As I'm approaching the summer, and I don't think I'll have a lot of work to do (other than reading up about genetic algorithms for my final year project, which I am psyched for), I'm looking to work on something. I could at least contribute a base for it.

I have had some practice with tokenisation, etc, but I don't know how well I could do this, it's always worth a shot, though.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on May 17, 2014, 02:10:22 PM
I'm doing a pass on the DNA code right now around the parsing and giving "compile errors".  You'd need to figure out syntax highlighting and saving/loading and anything else that makes sense.

Also it would be nice if most of it was set up as a control instead of a form so I could embed it in the program irksome in like the options window.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on May 17, 2014, 04:01:22 PM
How do you write a standalone control in .NET? I thought it had no component object model.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on May 17, 2014, 06:27:03 PM
Found it.

Quote
The control we will create will contain a text box for your name and a button that will fire an event. To begin, open Visual Studio .NET and begin a new C#  Windows Control Library. You may name it whatever you like, for this sample the project name will be SubmitButton.

 May be useful for noobs. here (http://www.akadia.com/services/dotnet_user_controls.html)

Then you just add it to your ".NET components" and it computes the control.
Title: Re: Programming task (DNA editor)
Post by: Panda on May 18, 2014, 06:15:14 AM
Right, I'll bear this in mind and I'll get back to you in a week and a half (after exams and after-exam alcohol).

EDIT: I forgot to say that I do not know anything about the syntax that we're going to use for DB3. Anything to help me with that?
Title: Re: Programming task (DNA editor)
Post by: Numsgil on May 18, 2014, 04:37:13 PM
I'm redoing a lot of it anyway. By the time you'll decide if you want to play with it I'll have a language spec written out I think.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on May 20, 2014, 04:15:20 PM
Numsgil, how exactly will codules and chromosomes work when they need to be crossed over?
Will it search for codules under a specific chromosome and add them to resulting robot? In that case will it remove duplicate codules? Also, what happens in situations if both robots have only 1 chromosome?
Title: Re: Programming task (DNA editor)
Post by: Numsgil on May 21, 2014, 04:34:06 PM
I haven't gotten that far, but probably codules will get appended one after another to form a single long basepair sequence.  Then, I'll determine chromosome matches by finding pairs with the largest common subsequence.  Paired chromosomes will be crossed over, and the pair split between daughter cells.  Unmatched chromosomes will be randomly assigned to one of the two daughter cells.  Though I might want to do something else to handle cases where a haploid cell tries to sexually divide.
Title: Re: Programming task (DNA editor)
Post by: Panda on June 27, 2014, 10:42:02 AM
Riiiiiiighhhhht. I'm back now and I'm still willing to do this. :)
Title: Re: Programming task (DNA editor)
Post by: Numsgil on June 28, 2014, 07:33:54 PM
Okay!  I've basically got the DNA in a good state.  First step is you'll need to get the code and make sure it actually compiles on your machine.  I've updated the Darwinbots3 (http://wiki.darwinbots.com/w/Darwinbots_3) page on the wiki.

Assuming you can open and build Sunweaver.sln (the solution file for the DNA), you just need to call Sunweaver.Parsing.Parser.ParseDNA to turn text in to DNA.  Importantly there's an overload that returns a list of CompilationMessages.  These messages have things like line and column numbers for syntax errors in DNA, and a short message describing what's wrong.  So first step is to get Scintilla to send text through that ParseDNA method, and provide red squiggly lines for syntax errors.  The parsing should be fast enough that you can run it on a background thread whenever there's a pause in typing (similar to what Visual Studio does).

After that, syntax highlighting would be good.  You can get a list of known sysvars and commands in Sunweaver.DNASystem.Sysvars and Sunweaver.DNASystem.Commands.  Anything else is stored in the Sunweaver.Chromosome you get back after you parse DNA.

After that, stuff to let you "go to definition" for codules would be neat.  But I'm not sure how that'll work.  At some point you might need to dig in to the code for parsing DNA and get your hands dirty.  Hopefully it makes sense.  And of course, the unit tests are good for figuring out how something is supposed to work.

Once you have a simple prototype working I'll help you get the code in to the repository in a way that makes sense.
Title: Re: Programming task (DNA editor)
Post by: Panda on June 28, 2014, 09:16:25 PM
I will get back to you on Monday about whether I've got everything working on this machine but I don't think it will be a problem. I know I have licensed version of VS2013 Ultimate on this machine so that definitely won't be a problem.

What extent of Unit Tests will you expect from me?

Just to condense it:

I'm pretty sure I have SVN rights but I cannot remember what my credentials are? I know I can still download the source anyway so that's not a problem.

Anything else that I'm missing?
Title: Re: Programming task (DNA editor)
Post by: Botsareus on June 29, 2014, 12:22:14 PM
This is a related repo. question:

When me and Panda where working on chloroplasts in DB2 together, every time another person made change in the repo. you had to create a fresh local copy. Very inefficient imo. I am guessing there is improvement on that in the DB3 repo. as you say " I'll help you get the code in to the repository in a way that makes sense." How does that work?
Title: Re: Programming task (DNA editor)
Post by: Panda on June 29, 2014, 01:59:56 PM
I can't remember having to make a fresh local copy whenever there was a change in the repo. :s Maybe we did but that was a while ago.

I guess what he means by that is we just find a logical place for it to go in the repo later rather than trying to sort it out now.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on June 29, 2014, 03:14:40 PM
Sometimes SVN bugs out but usually you just need to "Update", and it'll pull in all the relevant changes and merge it with your local copy.  If two people are working on the same file, it can cause conflicts that you have to resolve, and that's a huge PITA, but otherwise it's usually painless.

...

I don't expect you to test Scintilla, and the DNA stuff should already be unit tested, so you might not end up writing any tests at all.  If you find yourself writing non trivial amounts of code that are more than bridge code, you should probably test it.  Regex expressions fall in to this, if that comes up.  Either way I'll do a code review with you once you have something, so you can be sure that I'm happy with what you've done and it makes it in to the final product :D
Title: Re: Programming task (DNA editor)
Post by: Panda on June 29, 2014, 06:59:34 PM
Ooooooooo, it'd be nice to have my code looked at by somebody who actually cares about it. :D
Title: Re: Programming task (DNA editor)
Post by: Panda on July 06, 2014, 12:38:59 PM
It looks like everything is working. Sunweaver compiled fine. I'm going to get started now.

EDIT: Oh, and do you have a specific design pattern that you want the GUI to be created with?
EDIT2: What is the best way to reference Sunweaver?
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 06, 2014, 05:01:36 PM
To reference it, add a reference to the assembly in bin/release.

Use your best judgement in terms of the look-and-feel of the GUI.  When you have something we can iterate on it, but I don't have a good idea of what just yet, honestly.

Happy coding :D
Title: Re: Programming task (DNA editor)
Post by: Panda on July 06, 2014, 05:44:21 PM
Well, I have it highlighting when there is an unknown token but I'm unsure how I'm going to do the others.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 06, 2014, 06:41:59 PM
I don't know much about what Scintilla does and doesn't do.  But basically, for compile messages, I'd wait for the user to pause typing (maybe something like 1 second after the last key stroke), and then spawn a helper thread that uses the parsing in Sunweaver to build the DNA, and then when the thread is down you can do the red squiggly lines for an error.  I'm not sure what that feature is called, though, so I'm not sure what to look for in the Scintilla documentation.  Also you might try posting questions on stack exchange or possibly a Scintilla forum if there is one.  Often times just having someone who knows a library point you in the right direction is hugely helpful.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 06, 2014, 06:55:24 PM
Sorry, I didn't make that clear, I've basically done that for everything but the __anon# and other things. Think I have it sorted, though.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 06, 2014, 08:13:50 PM
Ah, grand :D
Title: Re: Programming task (DNA editor)
Post by: Panda on July 07, 2014, 07:37:54 AM
Scintilla requires some of its dlls to be in the search path, do you know of any way to get around that otherwise getting it working easily with SVN will be a bugger?

EDIT: I think I've got the basics of Syntax Errors and Sytax Highlighting down, unless you want the different commands separated, but I'm unsure how I'll do that yet, other than doing it all manually.

I'm not going to do the "go to definition" as of yet as that is just going to be a headache and I cannot be bothered with that. XD I'm going to get the other basic features like saving and loading, multiple files open in a tabbed view, getting the error message list to jump to messages, and some basic customisation.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 07, 2014, 10:44:50 PM
Are they native DLLs?  That always gets gross.  Still we can probably set up a post build step to copy the native dlls to bin/release along with the finished executable.  If they're managed DLLs, we can just add references to them to the project and I think Visual Studio will handle copying them to the destination directory.

When you have something in a good state, zip up however you've set it up (code and all), and I'll figure out where to put all the bits and bobs.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 08, 2014, 03:21:40 AM
Yeah, that's probably for the best, I cannot add 2 of them as references, unfortunately, so I guess they're native. I'm not very good at all of this Windows stuff. XD I currently have it

I have quite a lot of refactoring to do as everything is just going into one class while I'm getting used to it. I mean, it is mvp but I don't like the state it is in.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 17, 2014, 09:26:37 AM
Are we going to use any alternate file extension?
EDIT: I'm trying to do a tab control but creating a close button next to each tab seems like it will take a while to do it, so I'm just going to leave it is: with the context menu and shortcut.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on July 17, 2014, 04:03:58 PM
You should be able to make a tab control and then in the whitespace under the control to make a close button. At least if you are thinking about the kind of tab control that I am thinking about.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 17, 2014, 06:12:24 PM
Well, I'm thinking about Notepad++ sort of tabs, if you understand me.
Title: Re: Programming task (DNA editor)
Post by: Botsareus on July 17, 2014, 07:31:12 PM
Oh, I get it. You have to make each tabs contents as a nested object the belongs to the general tab object. The general object will also have a close button. Yea, that is very annoying.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 18, 2014, 01:26:13 AM
Are we going to use any alternate file extension?

I dunno.  Maybe .dna?

I'm dying to see what you have.  Let me know when I can look :D
Title: Re: Programming task (DNA editor)
Post by: Panda on July 18, 2014, 05:03:11 AM
I think you may be able to next week, depending on my wireless situation. It should all be fine, though.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 18, 2014, 06:15:40 AM
Oh, I get it. You have to make each tabs contents as a nested object the belongs to the general tab object. The general object will also have a close button. Yea, that is very annoying.

Hence leaving it for now. It's something that is less important.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 19, 2014, 10:47:14 AM
Right, I think it's at a point where I could get feedback. It's not polished at all, (and neither is the code if you want to see that) but there's something there. :)

EDIT: it would help if I actually uploaded the files.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 20, 2014, 10:25:37 PM
It either crashes for me immediately or nothing happens :(  Can you zip up your solution and I'll try debugging it?
Title: Re: Programming task (DNA editor)
Post by: Panda on July 21, 2014, 04:07:11 AM
I didn't get chance to test it on another machine so I guess it's missing some library or something. I'll zip the solution now for you.

I had it in the Modules folder of the DB3 solutions.
Title: Re: Programming task (DNA editor)
Post by: Panda on July 25, 2014, 03:15:22 PM
Have you had any problems with it at all?
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 25, 2014, 06:53:28 PM
Sorry I was out of the county most of this week. I'm going to try and look at it this weekend.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 26, 2014, 08:47:55 PM
Got it working.  It was an issue with x86/x64.  You had it set to Any Cpu, which on a x64 machine tries to run it 64 bit.  But some of the assemblies are x32, which it can't load in x64 mode.  Switching the platform configuration to x32 fixed it.  I'm playing with it now.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on July 26, 2014, 09:42:10 PM
Okay, I like this a lot :D  These are some nitpicks, but nothing too hard I don't think:

1.  Syntax highlighting doesn't seem to work until after you move to the next word.  eg: "5 4 store"   store won't hilight until you press enter or space
2.  Is it possible to have the error message for a error squiggle in the text editor show in a popup tooltip type thing if the user hovers over it?
3.  View->Show line numbers doesn't seem to do anything.
4.  The undo/redo stack seems too coarse to me.  In Visual Studio undo won't undo more than a single word at a time, but this is set up to undo an entire sentence or paragraph at a time.  This is probably more a matter of taste than anything.
5.  The error window and the text in the text editor will flicker when an auto-parsing happens and tries to refresh them.  I'm not sure how to prevent the flicker.  You'll have to play nice with the way winforms refreshes, and that's a whole can of worms.  So maybe leave this for last.  But there must be a way to update a control without it flickering.  You might need to google around or Stack Overflow to figure it out.
6.  Save as doesn't change the name of the tab.
7.  Probably just name the tab the document name and not its full path, unless possibly there's a duplicate tab with the same name open.  And then maybe you show only the file path that's different and not the same.  eg: robots\robotA.txt and robotsbackup\robotA.txt instead of c:\Darwinbots\robots\robotA.txt and c:\Darwinbots\robotsbackup\robotA.txt
8.  It would be nice if you had something in the tab to show if the file is the same as on disk or not.  At its simplest, you could do a blue circle next to the name in a tab to show that it's the same as on disk, and a red circle to show it's different.  Compare what notepad++ does.
9.  The find/replace window seems to flicker when it's opened for the first time.  It's probably loading all the controls up fresh every time.  Try creating a find/replace window at startup and just show/hide it.
10.  It would be nice if F4/Shift+F4 went to the next/previous error in the error list.  This is how Visual Studio does it.
11.  Likewise, keep the currently selected error hilighted when you switch focus back to the text document after a user double clicks on the error.
12.  Make the entire row in the error list clickable.  Right now, if you try to double click anywhere right of the end of the description text for an error, nothing happens.

...

I did a cursory glance at the code, but didn't take any notes.  There's a few rough places, but generally it's fine.  Also, when I try to double click on eg: DevControl.cs, the form designer won't show because of an error.  It looks like an issue with Visual Studio and not the form itself, as the code looks fine.  Does it happen for you?

...

Also it exposed a bug in my DNA parsing code, so if you were trying to figure out why code like "{ 5 4 add }" was erroring out, that's my bad :/
Title: Re: Programming task (DNA editor)
Post by: Botsareus on July 27, 2014, 07:58:29 PM
Wow Numsgil, you really are a tough debugger. Almost reminds me of my last boss.  :)
Title: Re: Programming task (DNA editor)
Post by: Panda on July 28, 2014, 12:34:13 PM
Wow Numsgil, you really are a tough debugger. Almost reminds me of my last boss.  :)

I quite like it, to be honest. :D I love feedback, especially negative feedback.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on April 01, 2015, 12:32:32 AM
Okay I've submitted what you sent me to SVN.  They're both added to Sunweaver.sln listed as Sunweaver.Editor and Sunweaver.Standalone.

It would be great if you could look at some of the things I mentioned.  Otherwise I'll eventually get around to them :)
Title: Re: Programming task (DNA editor)
Post by: Panda on April 03, 2015, 04:07:34 PM
I'm sure I will get around to them! It'll be in a couple of months before I'll have the free time to do it.

After that I hope to get on with something else for DB3, if there's anything I can do, but I can't promise anything.
Title: Re: Programming task (DNA editor)
Post by: Numsgil on April 03, 2015, 04:25:20 PM
I have some ideas for projects in Sunweaver if you aren't sick of it yet (things like static code analysis to find sections of DNA that don't do anything and remove them when executing the code to make it faster) .  Or if you want something more adventurous I might have some ideas for things in other modules.  At the easier side it's writing tests to expose bugs in things I'm pretty sure have bugs in them, and then trying to fix them.  At the more extreme end it's things like integrating Yeppp! (http://www.yeppp.info/) in to the math library and seeing what sort of performance gains that gives.