Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - rwill128

Pages: 1 [2] 3 4 5
16
I'm not sure. Hope he's doing well though.

17
Off Topic / Back again with an interesting machine learning idea to discuss
« on: November 19, 2013, 11:59:29 AM »
Hey everyone,

It's been a while since I posted around here, but at the end of the summer a bunch of crazy things happened in my life and I didn't have time to indulge my interest in programs like DarwinBots. Lately I've been working as a Java software developer though, which is an awesome transition from my previous work as a writer. It's nice to be back in the field even though I have an English degree.

Anyway, I've been doing a lot of research and reading academic papers on reinforcement learning techniques, and I was thinking about something Numsgil said when he explained that he thinks that artificial neural networks are more accurately described as weighted cascading gate mechanisms (or something similar). I completely agree, and in fact I only began to understand them more thoroughly when I stopped thinking of them as neural networks and instead thought of them as such.

But I was just brushing up on some math, and I realized it might be really cool (though not necessarily computationally efficient) to model an artificial neural network in a different way, based on something like real physics. (Not biochemistry or anything, just simple physics.)

Imagine a neural network where each node is modeled as a point in space (let's go with 2-dimensional space for now) from which a vector is extended down (toward the next layer of nodes). Data is transferred to the next layer by calculating the proximity of the previous layer's vector endpoints. (Or by many other alternative methods, perhaps, but this one makes the most sense to me.)

Training this kind of neural network could involve modifying the magnitude of these vectors so they end closer (or farther away) from the next layer of nods, which would be analogous to modifying the weights of a traditional neural network. But you could also modify the angle of the vector or even slide nodes on one layer along their axis so they are closer or farther apart. You could also give the nodes other kind of activity choices like pulsing with a greater degree of force, thereby affecting the next layer of nodes more strongly and from a greater distance. This would allow one node to trigger some kind of explosive activity in the network during specific event sequences while acting normally the rest of the time.

What you would have is an actual cascading network of nodes, where the data cascades through the network in a much more complex and multi-faceted (but still ordered) way than it does in a traditional NN that just uses weights and thresholds to decide how data passes through a node. My attraction to this idea stems from two things:

1.) It would be easy (and extremely captivating) to graphically illustrate this kind of neural network and watch it in action.

2.) Because each node interacts with other nodes in a mathematically complex way and dynamic way, it would allow for smaller numbers of neurons to approximate much more complex functions.

Anyway, I came here to post this because Numsgil usually has interesting thoughts on ideas like this.

18
Darwinbots3 / Re: Fluid preview
« on: July 09, 2013, 12:26:29 PM »
Same as Ammeh. :(

19
Off Topic / Re: First Paid Python Script!
« on: July 04, 2013, 04:15:53 PM »
New development. Thought I'd share with you.

Someone else found this paper online: http://www.acsu.buffalo.edu/~batta/iie.pdf

It looks like someone was wanting eLance freelancers to do their homework, or maybe even their job.

20
Off Topic / Re: First Paid Python Script!
« on: July 03, 2013, 08:38:18 PM »
I'll try out Guru perhaps at some point. I've actually been doing freelance writing work through a few personal connections and eLance, so I'm familiar with what you're talking about. Luckily, getting paid on eLance seems to be pretty easy so far.

You're right though, I think I'll try to negotiate a flat rate next time. These projects will take me longer than they should at first, but if I keep going with the flat rate it's much more beneficial as I get better. If I go with the hourly rate, I get paid less the better I get at my job! :-/

It's easier to negotiate a flat rate with writing because I'm familiar enough to estimate how long it will take. I'll just have to learn as I go with the programming. You're right though about people trying to add specifications onto the project after an agreement has been made. I'm usually polite and firm in telling them that they're changing the terms of the project, so I'll need to reflect these changes on the invoice. In the past that's worked out for me well (with the writing at least).

21
Off Topic / Re: First Paid Python Script!
« on: July 03, 2013, 06:26:14 PM »
I'm glad you agree that the problem was extremely strange.

He was apparently a civil engineer (whose first language is not English) who wanted a script to calculate the amount of time a person would spend in store as a function of how many aisles they went down and in what combination they went down the aisles.

It was a weird problem.

---

I explained my interpretation of the problem and also explained that the definition he gave wasn't clear in some ways. He said I had a basic understanding of the problem, and I asked him for more information because I didn't think the problem would be too complicated for me to solve once I had the specifics nailed down.

We exchanged a few more messages, and I got to the point where I asked him what the inputs of the function would be and what the outputs of the function would be (as well as how he wanted the output data to be stored). He seemed to get agitated at this point, and questioned my ability to provide a "professional" solution. I reassured him that I'd be submitting quality work and I also offered to lower my hourly rate, since I'm not very experienced.

Then he said that he'd give me a second chance, but his communications had been deteriorating for several messages now. He was impatient and not specific -- bordering on rude -- and he also said that if I took the project I would definitely not be able to bill him for more than an hour and a half.

So I decided not to accept the gig. But I've been really digging into the freelance writing I do right now -- mostly technical writing. Last night was the first time I tried to get hired for a freelance programming job. I figure I'll keep throwing out offers here and there because it's definitely something I'd like to get into. More fun (and I'll get paid more for my time, hopefully.)

---

EDIT: I think he mostly questioned my ability to do the work because I haven't customized a resume yet that really showcases my skills as a programmer. I also undersold myself by offering to work for so cheap. I basically didn't come across as being confident in my abilities, even though I thought I had the skills to do what he needed done. Not necessarily a big loss, though. He might have been a real bear to work with.

22
Off Topic / First Paid Python Script!
« on: July 02, 2013, 09:18:08 PM »
Hey all,

This is an odd question, but I've got my first chance to actually do some paid programming work (ever) and I have to know that I'm capable of solving the problem he's laid out for me before I accept the project.

I'm going to look like a complete math beginner here, but after looking over his description of the basic problem this function needs to solve, I still have some questions. I'm hoping you guys can give me some insight into what his instructions mean, because I understand them like 80% -- and I'm not sure whether the last 20% is missing because I'm not qualified or because his instructions are confusing.

-----

To solve this type of problem laid out in Doc1, even if it were much more complicated than the example, I'd still just write a function that sums the distance over various aisles (the "P sub M"s) for each combination of picks (i.e. each element of K*) and then picks the lowest possible summated distance (finds the lowest value of P). Correct?

As long as I know the probability of each aisle being chosen and he gives me the function or information I need to determines the distance for each aisle, it should be easy. The only inputs for the function would be M (number of aisles) and N (number of picks), correct?

I'm not sure I have everything figured out right, but I'm willing to work very hard and fast to get it figured out. (On that note, how does "h" feed back into the equation. It says to find E[P] for a given value of h, but then h doesn't reappear later in the description. I'm afraid I may be misunderstanding something.)

If you're able to give it a quick look over I'd really appreciate it.

Thanks

23
Off Topic / Re: Personal Project -- Biogenesis and QLearning
« on: June 28, 2013, 09:43:04 PM »
It's definitely an interesting idea. I think the parameters for behavior reinforcement should all be relative to the creature, though. So something like "this action benefitted my health, let's do it again" type behaviors. Access to global data like that seems like it would be cheating. Maybe that's not true though. I'll have to think about it.

Another thing I like the idea is the passing of several different possible values as rewards, and the use of genetic code in the organism to decide which one(s) will be used. It might be hard to think about it because the "code" in Biogenesis is much different than the code in DB.

But the idea is to allow the possibility for organisms to use a behavior reinforcement model that allows for modular and open-ended results. For example, one of the colored segments is "lilac," and it kills the organisms it touches instantly, but doesn't convert any of the organism's energy to health for the lilac-owning organism. I'd like for the behavior reinforcement model to allow the potential for an organism to develop a penchant for using their lilac segments, while that same creature's offspring might experience a mutation where their behavior reinforcement model encourages them to use some other segment, such as red one, which doesn't kill as effectively, but which converts the organism's energy into food.

If behavior were reinforced according to some arbitrary judgment of immediate benefits to "fitness," the red segment behavior might be more encouraged, but both could actually be "fit" behaviors according to the one true standard: whether they produce another generation.

I'll definitely let you know when it's all put together. I'm excited to see the results!

24
Off Topic / Re: Personal Project -- Biogenesis and QLearning
« on: June 27, 2013, 08:47:36 AM »
I'm thinking a reward strongly weighted toward "Did you gain more health than you would have through photosynthesis over this last click?" would work well.

But I was also thinking about passing a bunch of different parameters to the learning framework, along with a short genetic code that describes which ones to use as a reward (and with what weights). Then just let the evolution of the simulation figure it out. :)

Could end up with some cool effects... for example... altruistic behavior: a creature that "wants" the health of the organism it sees if that organism has similar color segments. Could be selected for naturally, too, I'd think.

25
Off Topic / Personal Project -- Biogenesis and QLearning
« on: June 26, 2013, 08:01:03 PM »
Hey all,

So I hope there's nothing wrong with me posting about another evolution sim program on here. If there is, I'll be okay with someone letting me know.

But anyway, I wanted to share the results of the project I've been working on for the last few days. It finally reached a milestone where I have a working "proof-of-concept" program, and now I want to tweak it so it looks prettier and has lots of interesting features. I've been using Biogenesis (Specifically, a mod of it called "Biogenesis Color Mod" : https://sourceforge.net/projects/biogenesiscolor/) and a QLearning framework I found at http://elsy.gdan.pl/

Check out some of the samples on the Elsy website. He's actually got a nifty little library there. He implemented a type of reinforcement learning that uses neural networks, and the "Wanderbot" or "Apollo Lander" examples both show what kind of tasks it's effective for.

Anyway -- I decided I wanted to combine Biogenesis, which I've always found to be a fascinating program because of its relative simplicity (but cool results), with this other guy's QLearning framework.

In Biogenesis, each creature's lines have different functions based on their color, and each creature's "genes" just store the overall shape of the creature. So more effective shapes are selected for. But movement patterns are largely unintelligent. But now, as of about 5PM today, I've got a working prototype of a Biogenesis mod where each creature is connected to its own (surprisingly effective) neural net, and can make movement decisions based on the information passed to that net.

---

With all the evolution sim enthusiasts around here, I thought I'd come and ask if anyone's experimented with either of these programs before, and if they have, do they have any input on this concept? Any ideas of how you'd like to see it implemented? How should creature's brains be given reward feedback? Should their "preferences" be stored as hereditary information?

26
Darwinbots Program Source Code / Re: Status of IM mode
« on: June 18, 2013, 11:10:15 PM »
Congrats! What are you working on?

27
Announcements / Re: DarwinBots 2.45.03 Full version is finaly here.
« on: June 08, 2013, 08:27:36 PM »
Hmm maybe those bots were just in my inbound folder from the last time I experimented with it?

Is there a network share or URL I can direct IM to?

28
Announcements / Re: DarwinBots 2.45.03 Full version is finaly here.
« on: June 08, 2013, 08:18:28 PM »
The first time I fired up Internet mode, a bunch of bots came in and then it said I needed to turn on pond mode. So I did.

Now no bots are coming in, and Internet Mode keeps telling me that it doesn't recognize this version of DB. That seems like it would be expected, but is that a problem?

29
Darwinbots Program Source Code / Re: Status of IM mode
« on: June 04, 2013, 10:18:13 PM »
This is so exciting to hear people talking about getting Internet mode active again.

It seems like the chances of really cool evolutionary dynamics/occurrences are going to be much, much higher if we can get a lot of people contributing to a larger simulation.

30
You're right that you could implement them in the code. It's just that when you do that the entire system is subject to mutations that are likely to break the code.

I was thinking of a system that allows variables in the genetic code that would affect various behaviors of the bot to be subject to the training you mention, rather than simply being static.

That way you've got evolution affecting the structure and logic of the code for each bot, with the neural nets adjusting behavior within an individual bots' lifetime.

You could implement that with DB code, but I was thinking more of a system that allows all DB bots to not only evolve but -- within some significant limitations, granted -- adjust their behavior during their own lifetimes.

----

Maybe neural nets aren't the best tool for that, but maybe they are, I'm not sure. I just like the idea of each bot having a brain -- the framework of which is set by genetic code.

Pages: 1 [2] 3 4 5