Came to me after watching the robots Jez linked to. I just want to write it down before I forget.
Basically, you'd have a controller veg that gets fed basically infinite nrg. It would create insanely large amounts of both slime and shell, to prevent it from being a viable food source. Poison too. It would be programmed to send info shots to bots that it sees that have certain criteria, forcing them to reproduce. Maybe this threshold value changes over time as the number of bots in the sim increases/decreases.
Another dummy veg species would exist as little more than food pellets.
Another controller veg species would act as the dynamic predator that Eric has mentioned. It would get stronger or weaker depending on how many animal bots it sees.
The animal bots would have rudimentary behavior to feed from dummy vegs and move around a bit if there isn't any food. It would seek out the controller veg when it feels that it's time to reproduce.
The hypothesis I have is that cooperation becomes easy and natural when you're reproduction is handled by another, independent agent. I think this sort of environment should foster cooperation as opposed to cannibalism.
The hard part is deciding the criteria that the master controller veg uses to decide wether or not to reproduce a bot. I'm thinking possible candidates are either age, nrg and/or body, or kills. Another option is maybe having the bots give nrg to the master controller as a sort of "bribe".
I'm betting that the bots won't learn to independantly reproduce very quickly. Though I might have to add some new features to ensure that only the master controller allows reproduction. I'll probably need to add a few features here and there anyway to get things like my predator up and running.