I definitely agree with this. Performance-wise, I expect it to be terrible to model each individual neuron as an object. The logical unit (IMHO) should be a layer of neurons, with matrix weights, vector biases, and vector output. Similarly, I think you'd be better off keeping the bias as a separate value, rather than concatenating a 1 to the input vector. I know it's what they do when presenting the math, but it means you'll be allocating a new vector each time through. Finally, I suspect that you'll eventually want to move the learning rate (and maybe even the learn methods) out of the neuron and into a dedicated "training"/"learning"/"optimizing" object. Perhaps overkill for the perceptron, but for a multilayer network, you definitely want to control the learning rate from the outside. I've been working in TensorFlow, so my perceptions may be a bit colored by that framework. Cheers, Johann On Wed, Apr 12, 2017 at 10:45 AM Ben Coman <btc@openinworld.com> wrote:
On Wed, Apr 5, 2017 at 8:27 AM, Oleksandr Zaytsev <olk.zaytsev@gmail.com> wrote:
Hello!
Several weeks ago I've announced my NeuralNetworks project. Thank you very much for your ideas and feedback. As suggested, I wrote examples for every class and tested my perceptron on linearly-separable logical functions.
I have just completed a post about my implementation of a single-layer perceptron: https://medium.com/@i.oleks/single-layer-perceptron-in-pharo-5b13246a041d. It has a detailed explanation of every part of the design and illustrates different approaches to implementation.
Please, tell me what you think.
Are my class diagrams correct or did I mess something up? Is there a design pattern that I should consider? Do you think that I should do something differently? Should I improve the quality of my code?
Yours sincerely, Oleksandr
Hi Oleks,
(sorry for the delayed response. Saw you other post in pharo-dev and found this sitting in my drafts from last week.)
Nice article and interesting read. I only did NeuralNetworks in my undergrad 25 years ago so my knowledge is a bit vague. I think you design reasoning is fine for this stage. Down the track you might consider that OO is about hiding implementation details. So just a vague idea, you might have SLPerceptron storing neuron data internally in arrays that a GPU can process efficiently, but when "SLPerceptron>>#at:" is asked for a neuron, it constructs a "real" neuron object, whose methods forward to the arrays in theSLPerceptron.
cheers -ben