Author Topic: How to report a bug  (Read 2941 times)

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
How to report a bug
« on: November 26, 2007, 11:56:12 AM »
What follows are some Best Practices for posting bug reports.  Following these recommendations will help the devlopers do a better job and may help you determinine if what you are seeing is actually a bug or not before you post it.  In the end, if you are in doubt as to whether what you seeing is actually a bug or confused by these recommendations, post it anyway.  We would rather see some incomplete information than miss something.  

First, is it really a bug?  Do a little research before posting a bug.  How is it supposed to work?  Have recent design changes changed how you think things should work?  Could it be a problem with your DNA?

Do some investigation before posting.  Find out as much as you can about the problem and put it in the bug report description.    If the problem is with a specific sysvar or with DNA execution, take a look at the bot's execution via the bot property dialog.  Is the gene firing?  Is the memory location value what you expect?  If a crash, can you make it happen again?  How?

Try to narrow down the scenerio to reproduce the problem.  Does it happen every time or only in certain circumstances?  Can you demonstrate the problem with a simple gene or a specific sim?

Choose as specific a title for the bug post as possible.  Titles such as "My sim doesn't work" arn't nearly as helpful as something more specific like "Reproduction fails when .nrg < 100".

Attach a sim file.  This is important.  A sim helps the developers see what settings you are using and other aspects of the environment: other bots, shapes, costs, etc. all of which can have an impact.  If the bug is a crash, an error.sim should be generated.  We need that.

Include the program version in the bug post.   We have to know what version you are running.  The bug may have been fixed already in a newer version.

Regress the bug.  Bug reports go through a life cycle.  The developers will edit the bug title to indicate which stage in the life cycle the bug is in currently:

No changes to Title - Developers haven't had a chance to look at the report yet.
OPEN - Developer(s) have looked at the bug.  Looks like an actual bug or developers don't know yet.  Some additional information may be included at this stage.  The most common is:

[blockquote]NEED REPRO  This indicates the developer needs more information FROM YOU to reproduce the bug.[/blockquote]
RESOLVED - Developer(s) think the bug has been addressed.  There are several ways bugs can be addressed:

[blockquote]FIXED - A code fix was made.  In such cases, a version number with the fix will be indicated in the title.
NO REPRO  The developer can not reproduce the bug and attempts to ask for more information have not resulted in enough information to reproduce the problem.
WON'T FIX - This indicates the developer(s) agree this is a real bug, but one that won't be fixed.  It may be too complex or involved to fix or there may be other limitations with the development environment or the programmer's skills which prevent the bug from being addressed.
BY DESIGN - This indicates the behaivour indicated in the bug report is by design.  It is how the program is actually supposed to work and no code change is necessary or forthcoming.
NO BUG - This indicates the post doesn't actually indicate any problem.  Surprisingly, we do get these occasionally![/blockquote]

CLOSED - A bug moves to this state when the original bug poster has regressed the fix (verified that the problem has really been addressed) and agrees that the problem has in fact been addressed.  IT IS YOUR RESPONSIBILITY TO REGRESS BUGS YOU FILE AND EDIT THE TITLE TO INDICATE THEY ARE CLOSED once you verify the bug has been addressed.  If you feel the bug has not been addressed in the indicated version (or for example, if the developer resolved the bug "no repro" or "by design" and you disagree) you should edit the title and change "RESOLVED" back to "OPEN" and post a reply to the topic with an explanation.

Thanks for your help in inproving the quality of Darwinbots!
« Last Edit: November 26, 2007, 12:14:09 PM by EricL »
Many beers....