Darwinbots Forum

Code center => Darwinbots3 => Topic started by: jknilinux on April 26, 2008, 03:55:49 PM

Title: Release date
Post by: jknilinux on April 26, 2008, 03:55:49 PM
When will DB3 be released? It won't take years, will it?
Title: Release date
Post by: bacillus on April 26, 2008, 11:49:54 PM
I haven't had a look into the development of DB3, but I think it's via a repository system. The speed at which the project is done, therefore, depends on how many people are working on it. So if you want it to be finished sooner, get a thick book of C++ or whatever else the program is written in and chip away at whatever you understand enough so that the whole program doesn't disintegrate because of your "progress".
I would, but I'm too lazy to learn another programming language, and am running F1 and updating the wiki at the same time.
Title: Release date
Post by: Moonfisher on April 27, 2008, 07:08:38 AM
I wish it was C++, haven't looked at the source but I think I remember someone saying it's in VB...
I have nothing against VB, but haven't used it much and I love C++  
And yeah it sounds like they're using a repository, but realy any project where several people are working on it should use one.
I doubt they have any time estimates though, I think this is more of a hobby project for most people, so it's hard to know when things will get done.
Also I noticed the amount of new features has a tendency to grow faster than they can be made... so the release date probably won't be in sight till after a feature stop.
Title: Release date
Post by: goffrie on April 27, 2008, 10:32:30 AM
The current version of Darwinbots is in VB, Darwinbots 2.5 is in C++, and version 3 is in C#, IIRC. Confusing.
Title: Release date
Post by: jknilinux on April 27, 2008, 05:26:12 PM
Quote from: goffrie
The current version of Darwinbots is in VB, Darwinbots 2.5 is in C++, and version 3 is in C#, IIRC. Confusing.

Why change the language so many times? Just use Java for 2.5, 3, and everything after that. Java is popular, easy, has tons of built-in GUI stuff, and will give you compilation errors ALL THE TIME to let you know when you're doing something that might give you a semantic error (aka code is less buggy). However, C, for example, is hard, has no GUI stuff, and will let you access the 5,000th element of a 200-element-long array.

Sorry for the rant.  
Title: Release date
Post by: bacillus on April 27, 2008, 05:49:31 PM
I concur, Java would be the best option. The trouble seems to be that VB, C++ and C# are the only ones that most people seem to know. I tried writing a version of DB in Java once; if I get a decent UI working and I/O as well (my two weak points), I should have a version .jar'd and running in no time.
Title: Release date
Post by: asterixx on April 27, 2008, 06:46:24 PM
Wont the language it is written in ultimately depend on what sort of purpose the program will serve? If it is anything like DB 2.43, Im pretty sure that Java wouldnt work, which is perhaps why it wasnt selected in the first place. . . ?

I havent been using DB for very long so if Im speculating here, please enlighten me. And I agree moonfisher, C++ is wonderful!
Title: Release date
Post by: bacillus on April 27, 2008, 07:09:39 PM
I think Java is a very useful language when making complicated programs. Programs like Ruby and Python (Both of which are very powerful languages, by the way) are good when writing smaller programs, but Java has so many plugins from the OS community that it's easy to find what you need and implement it via abstract classes. Exhibit 1: The Java 3D class. Combined with the dozens of physics engines out there, and interpreters, it'll be easy to write a 3D version of DB. I actually wrote mine in Java, then put it all together using JRuby. The Java VM is built into virtually all Operating Systems, so is platform independent, meaning no different versions have to be used for Linux, Mac, Windows, Solaris etc.
Title: Release date
Post by: asterixx on April 27, 2008, 07:51:01 PM
OK! I clearly dont know the history of DB well enough to know where it came from and hence, where its going. I get the sense that there are a lot of ideas and reccomendations flying around though.
Title: Release date
Post by: EricL on April 27, 2008, 09:18:52 PM
The main line 2.4x code is VB6 because it's was originally written in that language by Carlo, the original author and inventor a long time ago, when Java was still just a type of coffee and no-one since then has gone to the effort to port it to anything modern.

Later this year, if I'm so inclined, I will likely port it to a modern version of VB to get threads and a decent network I/O package and a bunch of other things.  Why stick with VB?  Because I'm lazy and this is the easiest way to get those things.  I'm not up to porting to a completly different language, at least not this year and the advantages of doing so don't outweigh the costs IMHO.

The current engine is primarily systems code.  Graphics are a surprisingly small portion of the total code.  Someday it would be nice to leverage some hardware acceleration or a 3rd party physics engine, but if your waiting for me to go there, don't hold your breath.  I have no such plans.
Title: Release date
Post by: bacillus on April 27, 2008, 09:22:26 PM
A concise history of DB.    
Title: Release date
Post by: jknilinux on April 27, 2008, 10:37:39 PM
But Ericl, if you aren't going to port DB to another language, why does it say that DB3 will be in C#? And why will you be doing DB 2.5 in C++?
Title: Release date
Post by: EricL on April 27, 2008, 11:25:32 PM
Quote from: jknilinux
But Ericl, if you aren't going to port DB to another language, why does it say that DB3 will be in C#? And why will you be doing DB 2.5 in C++?
DB 2.5 is dead as far as I know.  It was an experiment Nums did for a class I think and never really saw the light of day.

He is now working on what he calls DB3.  If and when it ever becomes real, and by that I means gets to a point of critical mass, stability and features that people can use it and I actually come to beleive it can replace the current code base, I will stop advancing the current code base but until that time, I will continue to fix bugs, impliment requested features, improve performance and otherwise take the current code base forward as I have have been doing now for well over two years.
Title: Release date
Post by: Numsgil on April 28, 2008, 03:52:20 AM
2.5 is indeed dead.  It inherited too many problems from the VB source without solving enough of them to make it worth while.  The current code just suffers too much from code smell to make it worth while to port it directly to another language, IMO.  Eric maintains the old code base, which is just fine with me, since it let's me focus on DB3 without letting the program grow static and die.

There were lots of possible languages to code a new version of DB in.  I chose C# because it combined the WYSIWYG GUI creation tools of VB6 with a strong set of coding libraries (ie: the .Net framework), with the ability to somewhat easily link to libraries written in C.

As far as a release date, it depends entirely on the physics engine.  The 2.5/C++ attempt taught me that, minus the physics, the program takes less than a year to code from the ground up.  So in order to determine an actual release date, I need to know how long it will take to finish the physics engine, which I just don't know.  At this very moment, I'm working on the physics engine.  Darwinbots has some unique needs for physics, and after some failed attempts using two different 3rd party physics engines I've decided to bite the bullet and code my own.  This physics engine is called "Lodestone", and is coded in C++ using the Test Driven Development methodology.  It's about 10K lines of code at the moment.  I still need to write various collision detectors, joints, and work on the core algorithms for collision detection and resolution.  However my new job keeps me busy so I'm progressing at a snail's pace atm.  Hopefully the pace will pick up when I get in to the swing of things, but it might not.  You can see the SVN for the physics here (https://svn2.hosted-projects.com/Numsgil/Lodestone/).

When it's done I'll write a .NET wrapper for it (which I've done in the past, and should just take two to four weeks), and then code the rest of the game (which should take approximately ten months), and then it'll be more or less done.  Or an early bug ridden version anyway   So I'd estimate 12 months from when the physics engine is usable to when I'll have an early version released.  And at my present pace, I'd estimate 6 to 12 months to complete the physics engine.

I'd be more than happy to have  other people work on this with me., if anyone's interested.  Especially people who know C#, C++, and/or physics very well.
Title: Release date
Post by: Moonfisher on April 28, 2008, 03:54:45 AM
I don't like Java... not a big fan... with C++ I can choose to build something exactly as I want it to work, or I can use one of the millions of libraries for it...
With Java I HAVE to use some sucky lib function that's poorly dokumented and generaly makes litle sence half the time, or eats up resources...
Not a big Java fan... I can understand people want to use languages like VB and C# for GUI and simple features... but I'd build everything else in C++...
Java has no features that C++ doesn't... it just forces you to use them so people are more aware of them.
I never understood what the fuss is about Java... it takes like 2 seconds to make a destructor... and you'll be able to make sure there are no leaks...

Anyways I'm looking forward to DB 2.5... didn't have time to read everything in here, but just saw poeple talk about Java and I had to comment... don't use Java for DB !!! I'll take VB, C# and C++ over Java any day... I repeat there's nothing you can do with a Java library that you can't do with C++ library... the only difference is that Java comes with half a million libraries and with C++ you only use what you need. (And who needs garbage collection, that's for noob's )
Either way Java has nothing... nada... only poorly dokumented junk libraries that you have to use because you have no freedom or controll...

Being a litle harsh, but I can't take it when poeple compliment Java and definately not when it's as an alternative to VB, C# and C++ !!!

I'd go for C# for all the GUI and simple functions, and build all the heavy stuff as C++ libraries..... I think those should be compatible... never checked... just assumed...

Edit:
Ok maybe I was a bit hard on Java. It's just always bugged me whenever I've used it. I haven't experienced libs in Java that weren't available for most other popular languages. I don't know Ruby or Python but I hear good things, just never got around to learning any of them. I also don't know much about compatibility, but I have a hard time imagining that C++ libs would be less compatibe with Linux and such...
Just seems so heavy and troublesome to work with Java, but maybe I just don't know the laguage well enough.
Personaly I'd still go for C# and C++ libs...
Title: Release date
Post by: jknilinux on April 29, 2008, 06:00:40 PM
Also, maybe the best thing about Java is that it can run in your browser. Imagine being able to try-out DB without downloading it, or using an account system to save your sim and look at it and continue running it at the library, just by logging in. You can't do that with c#.
Title: Release date
Post by: goffrie on April 29, 2008, 07:41:08 PM
Quote from: jknilinux
Also, maybe the best thing about Java is that it can run in your browser. Imagine being able to try-out DB without downloading it, or using an account system to save your sim and look at it and continue running it at the library, just by logging in. You can't do that with c#.

IMHO, Java web applets are infernal.
In C#, you can have a program which can be compiled into a machine code executable. I haven't seen a viable solution for that in Java. I'd rather not have an interpreter for a program like this, even if it's very efficient.

Also, C# is portable with Mono. Please try to make it compatible - a native Linux Darwinbots would be nice

I'd still use C, although I can see some problems arising with complexity and coding time.
Title: Release date
Post by: Numsgil on April 30, 2008, 02:19:07 AM
I'm mindful of Mono and Linux, but I haven't ever played around in Linux, nor do I have a Linux install, so I can't guarantee anything.
Title: Release date
Post by: fulizer on April 30, 2008, 09:04:50 AM
java would be nice but then I cant run it.
I think.
I could just download it
but I dont have the inernet.
Title: Release date
Post by: gymsum on April 30, 2008, 09:57:32 AM
Rubla is better than Java or Python, but we're not making heirarachle web page browsing are we.

Anyways, I know physics, but nothing about prgoramming, if you would like me to test it out I could.
Title: Release date
Post by: jknilinux on April 30, 2008, 11:10:13 AM
Hi nums,

go to getpclinuxos.com . It is an extremely good, extremely easy linux distro, so just install it on a tiny partition of your hard-drive or on an emulator, or on a thumbdrive, and problem solved. Also, isn't java like THE_MOST_PORTABLE_LANGUAGE_EVER? No matter what machine you're on, java will run with no modifications. With C#, though, it sounds like you have to actually work to make it portable.
Title: Release date
Post by: EricL on April 30, 2008, 11:45:57 AM
You pay about a 20% perf penalty for managed code of any ilk.  All that garbage collection and memory management and range checking the runtime does for you is nice, but it isn't free.  This is one reason operating systems and database servers and similar engine code such arn't written in managed code.  

DB is mostly an engine, mostly systems code.  A portable lanaguage like C or C++ would be fine, but IMHO, you want native compiles to specific platforms for engine code.
Title: Release date
Post by: Moonfisher on April 30, 2008, 01:00:49 PM
I don't think garbage collection is worth 20%... although I'd immagine that number is a litle exagerated...
But generaly a destructor rarely takes any time to make... and it may take a litle longer to propperly manage objects and pointers, but it's also nice to know it's being done right (A good performance test should make it easy to fix any leaks you may have caused). And I guess debates about C++ vs Java is nothing new, and I bet there's tons of stuff I know nothing about. But I know you still have to be carefull in java not to make any pointer loops and other stuff the garbage collector can't handle.
But in the end, I find C and C++ to be a lot easyer and more intuitive than Java. I have a real feeling that I can do anything I wan't, and I can either make it myself or find a library that does it for me. I also have a hard time imagining that you can make as efficient code in Java without having to be as thorough as in C. In the end I just dislike the odd syntaxes and "wierd stuff" you need to do when using Java libraries, could just be a personal preference, but when I use a C library it makes sence to me, that's not always the case with Java...
Appart from that any larger framework I ever worked with has always been in C++, not that I've tryed a lot (Unreal Engine 3 and Kynapse), so it's easyer for me, and I think the performance and compatibility of C++ shouldn't be the issue...

A history of popular FPS game engines, noticed Crysis is in C++, as are most of them...
http://upload.wikimedia.org/wikipedia/comm...8/Fpsengine.svg (http://upload.wikimedia.org/wikipedia/commons/d/d8/Fpsengine.svg)

But I actualy started with Java and I still found C to be a lot easyer when I first tryed it... maybe it's just me... just seemed like you need to know half the lib functions for Java before you can use it effectively... and all the dev environments I tryed where awfull...
Title: Release date
Post by: EricL on April 30, 2008, 01:20:39 PM
Quote from: Moonfisher
I don't think garbage collection is worth 20%... although I'd immagine that number is a litle exagerated...
It's not, at least it wasn't a few years ago when I worked in the Windows kernal group.  Managed code does a lot more than garbage collection such as object sandboxing.  Very usefull stuff for allowing large numbers of people to collaborate effectivly on a large project or for running third party code inproc without security concerns.  Not much use for DB as currently designed.  If bots were code objects like in Terrarium (written in C#) then that would be another matter.   Features are great if you need/use them.  Their baggage if you don't.
Title: Release date
Post by: Numsgil on April 30, 2008, 01:37:27 PM
I think it's closer to 10% than 20%.  That's been my experience (ported a 3D app from C++ to C#, both using Ogre (the C# version used a managed wrapper), and the experience of most people I've talked to.  The actual number varies with whatever is going on, but 10% seems to be the magic number.  And of course only 1% of the code uses 99% of the execution time in any CPU bound project (like Darwinbots), so that 10% overhead is very easily reduced if you can write key parts of your code in C++ or assembly or something like that.  So I'm very specifically ignoring any issues with constant code overhead, and instead entirely concentrating on the program's Big O.

I don't know much about Java, so it was never really in the running.  What I like about C# are things like run time reflection (ie: easily determining the type of an object at run time.  You can do this in C++, but there's a whole bunch of limitations that prevent me from doing fun things like BasePair bp = new BasePair("cond") easily), the huge .NET library, semi-easily interacting with unmanaged code (ie: with a little bit of work I can link with a regular old C++ library), greatly expanded exception handling (including things like a stack trace and line number), etc.  Garbage collection isn't something I either need or don't need.  It has its perks and downsides, like anything else, but it's far from the deciding factor.  If you like C++, and you spend time in C#, I can't walking away without good things to say about C#.  It's just a lot of fun to code in, and specifically has addressed language issues with C++ (I'm looking at you explicit cast conversions (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1592.pdf)!).

gymsum, I'm actually getting pretty close to where a tester would be nice.  I'll create a topic in a month or so when I get that far (fingers crossed).
Title: Release date
Post by: Numsgil on April 30, 2008, 03:02:42 PM
Quote from: Moonfisher
A history of popular FPS game engines, noticed Crysis is in C++, as are most of them...
http://upload.wikimedia.org/wikipedia/comm...8/Fpsengine.svg (http://upload.wikimedia.org/wikipedia/commons/d/d8/Fpsengine.svg)

Video game coders tend to be about a decade behind the curve as far as adopting new languages or ideas that the rest of the software industry embraces.  At my job right now, it's an uphill battle to use stl (I'm not allowed to use any of the containers, for instance), and most of the code would charitably be described as C with classes.  OO design is lacking at best.  On the other hand, it's pretty much solely video games that are driving any of the computing advances on PCs these days.  So it's a weird dichotomy of super high tech hardware and super ancient software.
Title: Release date
Post by: Moonfisher on May 01, 2008, 09:21:24 AM
I was always under the impression that it was a performance and memmory thing. It seems easyer to optimize performance in C++.
But I'm not geting the impression they're behind the times though. Not sure C++ is to be considered ancient technology, it's an old language by now, but I can't imagine all these people are using it just because they know it better. It shouldn't take too long for them to learn C# if that's what they wanted to use...
Unreal Engine 3 just buildt their own scripting language for the engine, and generaly OOP seems to be rather important, you're not forced to make it OO but it invites you to do so, and most of the things you make will become a base for many different types (NPC's, weapons, projectiles, asf...) so you don't realy have a choice.
I'm assuming the reason they used C++ was because it was the language in which it was easyest for them to reach the performance they needed. If there was a better way I don't see why they wouldn't try it.
Title: Release date
Post by: Numsgil on May 01, 2008, 02:33:51 PM
Adopting C++ is a very recent development.  Halo, for instance, was coded entirely in C.  Super Nintendo games were coded in assembly.  Programming with good OO design is rare in the industry, usually you have old salty programmers who like to code down to the metal in C or assembly still.  Ostensibly it's for performance reasons, but IMO this is a good case of Premature Optimization (http://c2.com/cgi/wiki?PrematureOptimization).
Title: Release date
Post by: EricL on May 01, 2008, 02:40:24 PM
Also, codebases are old.  Word, Excel, etc.  have decades of history.  Porting is hard hard hard.   Code written in a specific language tends to stay in the language.
Title: Release date
Post by: Blacksmith on May 03, 2008, 02:48:24 AM
http://labs.developerfusion.co.uk/convert/vb-to-csharp.aspx (http://labs.developerfusion.co.uk/convert/vb-to-csharp.aspx)
Title: Release date
Post by: Numsgil on May 03, 2008, 03:46:46 AM
While switching between .net languages is so easy it's almost trivial, moving from VB6 (the current Darwinbots code base) to .Net is decidedly not.
Title: Release date
Post by: Moonfisher on May 03, 2008, 07:41:33 AM
1.  Make it work.
   2. Make it right.
   3. Make it fast.

Heh that's actualy how I end up working a lot of the time... even for seperate small features (But with less emphasis on making it fast)...
The thing is you often know if a feature will need to be expanded a lot and may be run very often, so building this with speed and OO in mind can help a lot...
You don't need to make everything perfect in every sence, but you can predict uses for the system and make sure your structure supports it and is easy to expand, and you can run some early performance tests and generaly keep performance in mind while building your feature... not pass any strings and such... try to stick to pointers and such.
Also if you're going to store something and you know it will happen very often then you may want to consider how you're going to decrease the file size before building everything....

I agree there's no need to optimize for something that may never be needed, but I also think it's important to think ahead or you'll spend too much time refactoring code... Sometimes making something work right will barely be related to just making it work, hence that step should be skipped IMO, and sometimes performance is a great issue and therefor you need to remake virtualy everything if you didn't have it in mind from the start...
So I agree that for a lot of features you can proceed this way, but there's also a lot of cases where you need to think things through before getting started. (IF you know that structure or performance can be an issue).

But where game engines are concerned... I don't think they're optimizing prematurely... I think they need to optimize anything they can... GTA 4 still has lagg at times, and so does virtualy any game that didn't cut away anything that could slow them down with extreme prejudice (Like Blizzard who only accept perfect features, so IMO they often end up cutting away a lot of the fun, atleast in WoW they did, all that was left was a MMO single player game where you could brag to your friends about the gear you managed to grind, and if your friends weren't on your served... then it was actualy no different than a single player game, except monsters took ages to die) (Anyway that was offtopic, I'm just not a big fan of MMORPG's where they cut out the R to favor booring things like grinding and quests like "Bring this lettter to someone you never heard about, and may never hear about again" or kill X boars of Y color... worst asset reuse I've ever seen, I was killing boars at lvl 1... and several times at later levels, and then the expansion came... and the first thing I killed in the new world... was a board... and it took me longer to kill it than when I was lvl 1... so by that logic my character spendt WAY to many hours getting worse at killing boars... lamest game EVER... wouldn't play it again if I got payed by the hour... well...)

Anyway just saying, game engines can always use better performance, and WoW is the worst wana be MMORPG I've ever played, less fun than pong... (To be fair I did have some fun with PvP, one thing Blizzard knows how to do is balance PvP, but you get tired of the 3 maps they had pretty fast.)
Title: Release date
Post by: Numsgil on May 03, 2008, 02:22:15 PM
Ah, see, you fall in to this mental trap of "the game needs to be fast, so I need to write fast code."  But the thing is, you only need to worry about speed like this if and only if:

1.  Your game is CPU bound (I'll be you $100 that GTA is fill rate bound, ie: it's limited by the number of pixels it can draw.), and
2.  The section of code you're working on is a bottleneck in the code.  Note, however, that you can only determine this last one if you actually profile the code.  If you can't say exactly, with a number, how slow the code you're working on is, then it's premature optimization.  Meaning you do not write the code to be fast up front.  You write it to be readable and understandable 10 months from now when you go back and refactor it.

All this said, choosing the right algorithm is of primary importance, and cuts way deeper than optimization.  If you write an O(n^2) algorithm, then you're screwed, right from the beginning, because the scope of that particular algorithm has an inherent limit.  If you bubble sort a list of 5 elements, you might feel justified because n is so small, and writing a nlogn sorter might be "overkill", but then that's where the stl and 3rd party libraries come in.

So my priority list when I code (and get my say) looks like this:

1.  Choose algorithms with low Big O
2.  Write tests to validate the results of my algorithm, and check for pathological errors.
3.  Write code to fulfill those tests by any means necessary.
4.  Refactor the code I just wrote to look pretty.
5.  Profile for performance issues if necessary.  Ignore performance issues if not necessary.

So as an end result I have a lot of code that is easy to follow, with good algorithms, and tests to ensure they work correctly, but that is written in an inefficient manner.  And at the end of the day I might refactor those inefficiencies or I might not.  It just depends on whether it's necessary at the end of the day.
Title: Release date
Post by: Moonfisher on May 03, 2008, 05:29:08 PM
The only time I saw GTA lagg was when a large amount of NPC's got hit and went in rag doll mode... so it would seem like it was just taking too long to process the physics and collisions of all the ragdolls....
But I think a lot of the time, if you're going to build on something you never know exactly how far you're going to take it, so when building a base it feels safer to optimize performace right away so you don't risk having to make a change later on that will affect all the expansions or inherited... stuff... (I'm feeling very intelctual right now)
I kind of had the impression that lighting and physics where the big bottlenecks though... and I don't think most people have a physics card so I would imagine that code optimization should help, even if it's not the main issue.
Anyway... I'm just an intern... what do I know...
Title: Release date
Post by: Commander Keen on May 03, 2008, 08:08:35 PM
Readability is more important than optimisation. I've got lots of unfinished QB programs which will never be completed because I can't understand my own code  
My VB6 programs have a much higher success rate, even if that may be because I learnt to pro

At the end of the day, PC's are getting fast enough that you don't need to optimise everything. When I used to program on my 386 laptop in QB, I found optimisation vital, and I wrote several test programs that demonstrated this. Now, with computers capable of calculating over 1,000,000 times faster than before, we don't need so much optimisation, making the use of interpreted languages like Java and Perl useable for serious applications.

Ok, that's my raving done. No guarantees for reliability, because I am an amateur programmer, and most of what I know is self-taught.
Title: Release date
Post by: bacillus on May 04, 2008, 02:20:16 AM
I had a look today at the DB3 code. I was struck by the simplicity of the DNA parser, I expected a huge chunk of unreadable code, not a huge chunk of readable one.  I still haven't figured out how to commit anything, not that I have time with exams so close and the F1 league to chip away at. Although I would make the suggestion to move the whole thing to Google Code if it's not too cumbersome.
Title: Release date
Post by: Numsgil on May 04, 2008, 04:18:44 AM
I try my best   I'm rather happy with the DNA code atm, though it still needs some features to be implemented, and I skipped writing tests for the top level code.  And I sort of went overboard with exceptions in the parsing code...

To commit you'd need SVN access.  anonymous access is read only.  I can set up an account if you're really interested.  Same goes for anyone.  It's all version controlled, so if you screw things up I'll just revert it

I've heard that google code isn't quite up to snuff just yet.  Or that's what my friend at work said.  I might look in to it and see if it would work better than my private SVN for this use.
Title: Release date
Post by: bacillus on May 05, 2008, 02:56:27 AM
Google Code is probably so good because it's public; no-one can sabotage your work permanently unless they were an evil hypnotist with way too much spare time, and you sometimes end up drawing out some brilliant ideas from some random guy who drops in a gem of code.
Title: Release date
Post by: Trafalgar on May 06, 2008, 01:14:52 AM
Quote from: Numsgil
What I like about C# are things like run time reflection (ie: easily determining the type of an object at run time.

Side note: run time reflection has poor performance. Delegates also have poor performance. But, yeah, don't worry about that much.

There are cases where you can write code which is faster without being less readable, and if you know about those you can take them into account when designing something. For example, if you're planning something which you would normally do (numFoos % maxFoos) (modulus), you can swap the slow modulus operation for an extremely fast & operation if numFoos is always positive and maxFoos is (2^n), where n is any positive number. You would & it by 1 less than you would have %'d it by. (numFoos % 4) would become (numFoos & 3), for example.

E.G. Here's an example in javascript (I just copied this right out of what I'm working on at the moment).
Code: [Select]
var blankSubEntry = (lastWritten[(lastWrittenStart+matches)&maxMatches]==="");
if (hash == lastWritten[(lastWrittenStart+matches)&maxMatches] || blankSubEntry) {

Now normally you might say "well I should store (lastWrittenStart+matches)&maxMatches into a variable and then refer to that" - and that's fine. But hopefully the compiler should be smart enough (but not necessarily with javascript...) to optimize those so that'll only be calculated once anyways. Even if it doesn't, and you do that 5 times, it should still be faster than doing it with % once. (% takes 30-40 clock cycles, IIRC, and & takes about 0.5, in a pentium 4*)

* = I'm oversimplifying how that works. It's actually more complicated than that, and depends on what other operations are being done at the time, and what's going to be done after it, and I don't have a clue how it has changed after the pentium 4 or with AMD CPUs, etc.

So, things I usually do:
1. Prefer & instead of %, and choose power-of-2 sizes for things which will be using it (otherwise we can't use &).
2. For distance checks, omit the sqrt, and premultiply whatever we're going to be comparing against. You can calculate xdist*xdist + ydist*ydist and compare it to somedist*somedist to determine if the distance is <, =, or > than somedist. You can't determine how far apart they are, though. (Sqrt is slow)
3. If doing foo squared, replace with foo*foo (an optimizing compiler might already do this)
4. If doing foo * 2, replace with foo+foo (an optimizing compiler might already do this). Since multiplication (30ish cycles?) is much slower than addition (.5 cycles), you could even do foo+foo+foo+foo if you really wanted to, except that foo << 2 would also work there and would probably be faster.
5. When designing things that will be multiplied or divided by a number, prefer designing them so that you can use power-of-two multipliers/divisors. The compiler might be able to optimize multiplication/division by a constant which is a power of 2 to use << and >> instead, which is faster than multiplication, or you can do it yourself.

I tend to code in an already-optimized-but-readable style, but leave the really heavy optimizations to the compiler (especially with c#). Most of what I do are things the compiler can't or wouldn't be likely to - e.g. it can't change the number of terrain types from 7 to 8 and change the % to & and such without breaking your program, but you can design it better instead.

(In some languages I tend to optimize less. For instance, javascript for firefox extensions. I once tried to determine what was the bottleneck in an extension of mine which ran rather slow on many webpages, only to track almost all of the CPU use into a part of firefox's javascript math crap that I couldn't do anything about. That is, the extension checks the HTML and CSS for a page and reverses colors to change dark-on-light pages to light-on-dark pages. The problem is that for some reason firefox's JS math was terribly slow, and the string to int stuff and vice versa too (which were pretty much required for the extension to work and be useful).)
Title: Release date
Post by: Numsgil on May 06, 2008, 02:17:54 AM
Quote from: Trafalgar
1. Prefer & instead of %, and choose power-of-2 sizes for things which will be using it (otherwise we can't use &).

That's specifically what I'm avoiding.  You see, it just doesn't matter if there's a 100x faster way of doing something if that function only gets called 3 times a second every other Tuesday.  Code for people and algorithms 99% of the time.  Then profile, identify hot spots, and refactor/optimize them away.  You might know how your code works, but it's going to be Greek to the guy that maintains it after you.

Once you crack out the profiler and are able to say something like "Function FooBar is taking 5ms per call.  Reduce that down to 2ms per call", you can iteratively refactor your code just for that function to achieve the desired performance.
Title: Release date
Post by: Moonfisher on May 06, 2008, 10:22:02 AM
That's a nice trick, I'll have to remember that when working in native code.
But I would definately write a comment with the "normal" way of doing it and probably mention the conditions for the values used.
Probably make a function that used that method when the parameters where met and otherwise the normal way.

I know you don't need to optimize everything to an extreme degree, but if you have a feeling the feature you're working on will be used frequently you may aswell optimize it right away. And often optimized code isn't much harder to read, you don't have to go completely nuts writing everything in assembly language, just managing memmory and not passing strings and such will make a significant diffrence and would barely make the code any harder to read... IMO pointers aren't that bad, just takes a litle getting used to with all the referencing stuff.
Ofcourse if a function is rarely used it's not going to make a significant difference, although if you have a lot of heavy functions they can add up.
But so far it seems to me that a lot of the features made in a game engine will get called very often when they're in use. Or if it's triggered by an event of some sort there's a risk it can happen too many times at once. I can immagine people getting carried away, but I guess any optimization will help if you're already straining the processor with lighting and physics and such... so it may actualy make a noticable difference in the long run, I'm not sure how far they take it, but I know some of the native code is way beyond my understanding and I don't think they did it for fun  (But I would imagine that when you want to sell a game engine it's good to optimize almost everything since you don't know how often people are going to use certain features.)
Title: Release date
Post by: Numsgil on May 06, 2008, 01:56:32 PM
Yes, there are certain features that you know up front need to be optimized, because you have prior domain knowledge (you worked on an engine before and this part was slow).  Global knowledge helps too (this function is inside a tight loop that gets called 1000 times every game cycle).  But even with these cases, if you write the code to be fast first, you're making a mistake.  The problem is that, trivial optimizations aside, you don't know for sure what's going to be fast and what's going to be slow.  Modern computers have all sorts of caching issues, math coprocessors, etc., that change from year to year and change what sort of optimizations you need to make for your hardware.  Optimizations always assume something about the hardware your using (it has SSE, or it uses bits, that sort of thing).

As an anecdote, there's a popular sqrt approximation that Quake used involving bit manipulations and some adds and multiplies.  Sqrt calls were a bottleneck for the Darwinbots Visual Basic code.  I naively replaced some sqrts with these approximations, and just assumed it would be faster.  I believe it was Sprotiel who actually profiled and showed me that my Quake approximation was actually slower than a native call to sqrt.  I don't know exactly why.  Probably modern processors have a hardware level sqrt function that just beats any software approximation I could write.

Moral of the story here is even if you think you know exactly how and where to optimize, use a profiler anyway.  At no point should you make a change for the sake of optimization without profiling before and after.
Title: Release date
Post by: Trafalgar on May 06, 2008, 05:46:08 PM
By the way, what do you use for profiling in C#? (Last time I looked, I didn't find any free or open-source ones)

Profiling without an actual profiler is a bit of a pain. Running a loop which calls a function several thousand times, and timing how long it takes, doesn't necessarily give you results which reflect the actual real-world performance of the function, as far as I could tell.
Title: Release date
Post by: Numsgil on May 06, 2008, 06:00:00 PM
Performance hasn't been an issue with any C# apps I've done yet, but I think the professional edition of Visual Studio (which I have) has some in-built profiling.  A quick google search also led me to this page (http://csharp-source.net/open-source/profilers).
Title: Release date
Post by: bacillus on May 08, 2008, 02:53:54 AM
Have you heard of a tool called Clover? It searches through the code and checks which code is not covered by test cases. This could be useful as there are an enormous amount of things to be tested, by the looks of it.
Title: Release date
Post by: Numsgil on May 08, 2008, 03:00:59 AM
Hmm, that does sound useful.  Have you used it before?  How easy would it be to set up?
Title: Release date
Post by: bacillus on May 08, 2008, 03:23:00 AM
I haven't actually used it, I just heard it's a good tool. I could get my hands on the Java version, then see if it's worth implementing the .net version.
Title: Release date
Post by: Numsgil on May 08, 2008, 01:35:09 PM
Only if it's easy to do   I wonder if there are any open source alternatives, too.  $800 is beyond what I like to spend on software
Title: Release date
Post by: gymsum on May 08, 2008, 02:44:14 PM
Quote from: Numsgil
Only if it's easy to do   I wonder if there are any open source alternatives, too.  $800 is beyond what I like to spend on software

Unless anyone knows Linux, C++/C are the most powerful languages to use. Java's nice, but eveything for programming comes standard in a REd Hat Linux CD, which is now open source I think.
Title: Release date
Post by: bacillus on May 09, 2008, 02:08:47 AM
I was able to see it in action yesterday. Clover can be run either as a standalone program or a workspace plugin; either way, you either load up or select a project, then explicitly compile and execute all the test cases. (This step shouldn't be hard assuming tests can be run in batches in C# as well as in Java). Clover will then generate an image/pdf/Text file showing all the code that wasn't referenced by the test classes.
 It looks like the commercial version is mightily expensive, but there is a free trial version that should do the job. Looking into Open Source alternatives as we speak.
Title: Release date
Post by: Peter on May 09, 2008, 05:25:24 PM
Quote from: bacillus
... and am running F1 and updating the wiki at the same time.
Heh, I saw this. You´re runing F1. What does that mean, as I checked the beastery there are still desame number of bots. What exactly do you do with the leagues.

Title: Release date
Post by: Numsgil on May 09, 2008, 05:40:52 PM
See this topic (http://www.darwinbots.com/Forum/index.php?showtopic=2609)
Title: Release date
Post by: bacillus on May 09, 2008, 06:18:51 PM
I hope it's not too late, but don't read the whole thing. Past a certain point eg. the first couple of posts, there is absolutely no information.

This is a post giving information about lack of information