Author Topic: Yet another overflow bug from Testlund  (Read 10171 times)

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #15 on: January 14, 2014, 04:25:13 PM »
I don't know the .NET package or optional components well enough to answer the question(s) actually. I would love to read Numsgil's take on this.

In regards to what specifically?

In terms of unit tests, there's some stuff built in to Visual Studio, but it's not available in the free Express version, so I've been avoiding them and instead using my own unit testing framework.

In terms of performance, I usually use slimtune for .NET projects.  It's not full featured at all, but it's the best for what I need to do (identifying hot spots).

Dude, I do not leave them as comments in the code.
By unit tests I mean I usually visually confirm that what I put in is happening:
Most of the time I just break the execution and/or throw break points and see what kind of values I am dealing with.

"Unit testing" has a very specific meaning in software development.  See this article series, for instance.

That isn't unit testing.  I don't think there's a name for that style, but it's what coders traditionally did in the past.  It's falling out of vogue (a few vocal holdouts notwithstanding) in favor of constant automated testing.  Basically proving the code works right now by showing that its output matches a predetermined known good value.

Offline Botsareus

  • Society makes it all backwards - there is a good reason for that
  • Bot God
  • *****
  • Posts: 4483
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #16 on: January 14, 2014, 04:50:30 PM »
Thank you for clearing it up Numsgil.  :)

edit:

Quote
Let?s take a moment to think about how you write code without using test-driven development. You decide you need to implement some piece of functionality, you mentally break the problem into smaller problems, and you start working on each of them. While you write code, your only guide for whether you?re heading in the right direction or not is whether your code continues to compile. Once you?ve assembled a minimum number of those smaller subtasks, your program might even be able to run. So you fire up the game or tool and ?see? if it works. If it?s not obvious (or if it doesn?t work at all), maybe you break in the debugger and try to step into the code you just wrote to make sure the program is doing what you intended. Once you see it do its thing, you happily commit it to version control and move on to some other task.

lol, that is exactly what I do.
« Last Edit: January 14, 2014, 04:55:09 PM by Botsareus »

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #17 on: January 14, 2014, 05:02:40 PM »
I don't know the .NET package or optional components well enough to answer the question(s) actually. I would love to read Numsgil's take on this.

In regards to what specifically?
Something like smoke testing if functions are resistant against junk as input. Although that's likely not useful for DB2. Much odds for false positives and junk may turn into destructive junk for other functions but not crash.

TTD useful to do, because it forces you to write code in a way that suits testing. Preaching this, yet I can't say that I've used the order of unit-test->code consistently...  :P

If you implement unit testing(sure it exists, but clueless on how it works in VB6) it's handy to write down future bugs as unit tests(that fail). So when you fix it you got a unit test that'll make sure similar errors will be seen.
Oh my god, who the hell cares.

Offline Botsareus

  • Society makes it all backwards - there is a good reason for that
  • Bot God
  • *****
  • Posts: 4483
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #18 on: January 14, 2014, 05:09:58 PM »
It is just hard for me to write tests using code. Usually my brain is the test. Otherwise it seems like a waste of time for me, example:

a + b = c

Do I really have to test?

Code: [Select]
if a = 2 and b = 3 and c = 5 then msgbox "pass"
Reading article further...

Offline Botsareus

  • Society makes it all backwards - there is a good reason for that
  • Bot God
  • *****
  • Posts: 4483
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #19 on: January 14, 2014, 05:24:13 PM »
Also, how am I supposed to write a test that is designed to fail? There is only one way that
Code: [Select]
a + b = c and infinite ways
Code: [Select]
a + b <> c

Offline Botsareus

  • Society makes it all backwards - there is a good reason for that
  • Bot God
  • *****
  • Posts: 4483
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #20 on: January 14, 2014, 05:46:00 PM »
ok guys here is an example of the last thing I ended up fixing. Please give me an example of how 'Unit Testing' would have helped me in the given situation because I still feel like I am missing something here:

The following code is for the Point2 mutations, the "Botsareus 1/14/2014" is where it crashed before.

Code: [Select]
Private Sub Point2MutWhen(randval As Single, robn As Integer)
  Dim result As Single
 
  Dim mutation_rate As Single
 
  'If randval = 0 Then randval = 0.0001
  With rob(robn)
    If .DnaLen = 1 Then GoTo getout ' avoid divide by 0 below
   
    mutation_rate = .Mutables.mutarray(P2UP) / SimOpts.MutCurrMult
   
    'keeps Point2 lengths the same as Point Botsareus 1/14/2014 Checking to make sure value is >= 1
    Dim calc_gauss As Double
    calc_gauss = Gauss(.Mutables.StdDev(PointUP), .Mutables.Mean(PointUP))
    If calc_gauss < 1 Then calc_gauss = 1
   
    mutation_rate = mutation_rate / calc_gauss
   
    'Here we test to make sure the probability of a point mutation isn't crazy high.
    'A value of 1 is the probability of mutating every base pair every 1000 cycles
    'Lets not let it get lower than 1 shall we?
    If mutation_rate < 1 And mutation_rate > 0 Then
      mutation_rate = 1
    End If
 
    'result = offset + Fix(Log(randval) / Log(1 - 1 / (1000 * .Mutables.mutarray(PointUP))))
    result = Log(1 - randval) / Log(1 - 1 / (1000 * mutation_rate))
    While result > 2000000000: result = result - 2000000000: Wend 'Botsareus 3/15/2013 overflow fix
    .Point2MutCycle = .age + result / (.DnaLen - 1)
getout:
  End With
End Sub

Before I simply had:

mutation_rate = mutation_rate /  Gauss(.Mutables.StdDev(PointUP), .Mutables.Mean(PointUP)) and the value was going negative resulting in a negative overflow.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #21 on: January 15, 2014, 02:30:50 AM »
It is just hard for me to write tests using code. Usually my brain is the test. Otherwise it seems like a waste of time for me, example:

a + b = c

Do I really have to test?

If you were writing the add function for a processor in machine microcode you'd absolutely want to test this.  Does you add function work on positive and negative numbers?  Does it handle 0 correctly.  Does it handle 1 correctly.  If you're using floating point numbers, does it handle infinities and NaNs correctly.  If you're just literally typing a + b you probably don't need to test that.  But then it's probably part of some larger calculation you should be testing.

Basically, if you're writing a closed algorithm, something that takes data in, does stuff, then spits an output, you should be unit testing it 100% of the time.  If you're writing UI, networking, graphics, etc. stuff it's a bit more of a gray area, because it's difficult to isolate your code from random hardware issues and constantly iterating designs.  For graphics, for instance, do you do a per-pixel check?  What if one of the pixels is a slightly different shade of blue?  Is that a failure?  But you should definitely be unit testing the smaller pieces that make up the larger whole even then.

...

For the example you have, you'd probably want to break the random away from the function and have the function take as input 1 or 2 random values.  Then you can test to see what happens when you pass in negative values.  Basically non deterministic factors are the enemy of unit testing.  You usually want to isolate them from as much of the code as possible to allow for deterministic testing.  Unit testing isn't just a matter of writing the tests, it often requires rethinking how to approach a problem in a way that's amenable to testing.  That's usually a good thing, as it forces you to write code in a more modular way and really think through dependencies.

It does take more upfront effort and thought, though.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #22 on: January 15, 2014, 01:10:55 PM »
Also, how am I supposed to write a test that is designed to fail? There is only one way that
Code: [Select]
a + b = c and infinite ways
Code: [Select]
a + b <> c
First write a test, test will fail, then write the code.
Oh my god, who the hell cares.

Offline Botsareus

  • Society makes it all backwards - there is a good reason for that
  • Bot God
  • *****
  • Posts: 4483
    • View Profile
Re: Yet another overflow bug from Testlund
« Reply #23 on: January 15, 2014, 04:45:24 PM »
That is some really good advice, thanks guys.
I will have to print this and read it over a couple to fully get it.