Chapter 4 seemed to be justifying the use of non-linear elements in the neural nets and explaining the math behind non-linear equations. I think that the authors assumed that the reader has had no math before and has never seen a curve and felt compeled to explain them to the reader. However, there were a few intresting coments made: sigmoid function keeps the nodes activation under control if the node gets a lot of activity, the output will peg one way or the other and that if the input is near the middle of the sigmoid curve, small changes in the input make big changes in the output.
other than that, this chapter was mostly a summary of math 16-math 30
I didn't find chapter 5 very intresting. The first half was a lot of neroanatomy, which was intresting just for information but I didn't really see how it connected to the rest of course. The second half was really hard to get through because it was so dense and had so much medical jargon. I was able to a feel for some of the capabilites of the brain but it didn't really seem to fit into the course.
Why should we need to know how the brain forms if we're not building the brain the same way? We're using sillicon, not carbon, so who cares if its put together differently, as long as it does the same stuff? The chapter was intresting along medical lines, but I don't really see how it fits with the rest of the course, except as a source of examples that we should stive to have our nets able to do.
It was the best of readings, it was the worst of readings. It was the most inane of readings, it was the most profound of readings. I am of course referring to the duality of this weeks assignment.
Chapter 5 provided some of the most complex, interesting, and (to me) very confusing facts about the brain and brain development. There was fascinating discussions of innateness. There were debates of protomap versus (I can't remember the term). There were enough facts to choke a camel (whatever that means), and more than you could shake a stick at (whatever). Chapter 4, on the other hand, was, to this former physics major, really annoying. It was this horrible, disorganized, ad hoc, and simplistic look at differential equations that included much more mystification ("Oooh! And now we get _two_ effects from _one_ equation! Oooh!") than substance. I guess the disorganization on these topics was consistent with chapter 5, but because of the dearth of material I noticed it a lot more in chapter four. Really, there was no grounding of any of this either in studies of ANN's of in neurological study, only the few measley examples of post-mortem analysis where you could fit "non-linear" functions to some processes. I did like that one example where the "two-stage" data on birdsong was better fit with the one equation. The chapter did have a point, but a couple equations and a few examples would suffice as opposed to pages and pages of poor explanation of PDE's.
I guess Chapter five had one of the same problems as four: disorganiztion. I really had a hard time taking stock of what it was that they were presenting in this chapter. There was a lot of material, and seemingly very few organizing principles to help us make sense of it all. I was fascinated by the lesion studies. I hope we will be doing something with lesioning in lab this week. I also really was intrigued by the connection between thalamic excitation and functional area. But what seemed equally interesting was the fact that despite a decrease in the area of a region handling a task, the book seemed to indicate that there was almost complete function in the significantly smaller area.
What I found most interesting in this reading was the discussion of children with Williams and Down Syndromes, and how the effects of these forms of mental impairment compared with the effects of focal injury to the areas of the brain held to be responsible for language processing. It is pointed out that children with Williams syndrome have fairly normal language performance, while having severely diminished performance in the realm of visual-spatial tasks, while those with Down syndrome seem to have severe language impairment. These contrasting accounts of mental disability seem to be reminiscent of adult accounts of left- vs. right-hemisphere brain trauma. However, MRI shows no evidence of such inter-hemisphere differences between brains with Williams and Down syndrome. Instead, brains afflicted with these syndromes show structural differences along the front-to-back axis instead of along the left-to-right hemisphere axis.
These accounts deal a significant blow to the commonly accepted belief of one hemisphere's necessarily having more to do with language than the other, and to the popular idea of right-brained vs. left-brained people as a way of describing personality traits (i.e. rational vs. artistic).
It would be interesting to see a case of an epileptic who also was a sufferer of Williams Syndrome. Often, those with severe cases of epilepsy will have their corpus callosum (the structure that connects the left and right hemispheres of the brain) severed in order to curb the spread of seizures from one hemisphere to the other. This operation causes certain behavioral anomalies in performing tasks that involve both sides of the perceptual apparatus. For example, a patient's right eye will be covered (which connects to the left side of the brain, the side thought to be responsible for language), and she will be asked to give the name of an object shown to her through the left eye. This will be difficult or impossible for the subject with the severed corpus callosum to do, seemingly because one side of the brain can no longer talk to the other side of the brain directly. If this same patient also had Williams Syndrome, would the severing of the corpus callosum cause other, more bizarre behavior anomalies because of the presumably atypical arrangement of the patient's brain?
Many of the examples presented in chapter four of the ways that inherent properties of neural nets (such as a decrease in change as learning progresses) can reduce the number of extra factors that must be considered to affect the model were perfectly good although they seemed to simply be going over ideas that had been presented before, albeit more formally. But in some examples I think they went too far and started to get into the shady areas of statistics. For example, with the graph of bird song acquisition I would have to side with the biologists who have actually studied the birds. There are many other examples of "programmed" learning or behavior in birds which cannot just be explained by the internal learning processes of a neural net (especially in the case of programmed behavior).
For the second part of the chapter, which dealt with dynamic systems, I wish that they had gotten a little more hand-wavy and explored some of the possible implications of chaos and attractors. There's gotta be some good fringe stuff there.
My interest in the attractors, etc, is sort of carried on in the next chapter, where the authors point out that the brain is a different machine running a different program every time you learn something. Not only that, but the brain is a different machine running a different program every time you open your eyes or listen to a noise. The entire system is affected immensely by the information coursing through it, and judging from what happens to people in sensory deprivation experiments normal input may be essential to normal thought. It seems incredible, but the brain appears to be able to use the order of the world (rare as it is) as a source of stabilization and grounding.
I'm very interested in the bit where the various sensory centers of the brain "talk back" to its sources of information. Beyond input modulation, the first thing that springs to my mind is that part of the information passed back could be the sensory cortex's reconstructed version of the input (a la ALVINN), which could then be compared by the input source to the original and then used as a source of correction and training.
In the area of additive events, I would like to point out that recent research has been discovering that many more neurons are born and grow new connections in later life than was previously supposed.
I was unconvinced by figure 4.9. According to the text, the two state linear model accounted for only 60% of the variance, while the single state, non-linear model accounted for 73%. Yet, this is not really supported by the graphed data. Although the solid line appears to match the data a little bit more closely to the data from 100 days on, it would be much more convincing if there were data points at around 50 days between 50 and 20% correct song learning.
This particular example (birds) got me wondering a couple things. The first is that while most birds seem to imprint birdsongs when very young, there are some (eg., mockingbirds, catbirds) that I believe continue to learn other birds' songs as long as they live (incidentally, it is the male with the most songs who gets the females). How is that accounted for in the RI model, which doesn't seem to allow for much sensitivity in the long term? I don't think that the brains of mocking/catbirds are so different that their capacity for song learning would be completely different from near relatives. Also, I think (tho I need to look this up) that birds that have imitation abilities are scattered throughout the phyla, and not confined to just one lineage. Thus, it seems unlikley that their brains operate in a completely different way.
Migration is another thing that is generally thought of as being imprinted and static, even innate. Yet some birds have the capacity to change their migration pathways, even at an advanced age. Thus, a bird migrating south who is blown east by a storm does not necessarily continue to fly south, but can compensate and fly instead southwest. Thus, even though one might suspect that migrational behavior is imprinted, it appears that the organism is able to change its behavior/imprinting if necessary.
Therefore, I think that they have a valid point in their discussion of a period of sensitivity. I just think that it may be more complex than the simple axiom that 'we learn better when we are young.' And even though they center most of their thought on mammals, especially primates, it is interesting to think about the so-called lower species, who rely greatly on innate/instintive/genetically-coded behavior and how experiencial learning can change even these most hard-coded of behavioral models.
the differential equations review was useful (i have an exam tomorrow morning), but i'm not really clear on what the relationship is between activities that exhibit nonlinear growth characteristics and the use of a nonlinear (sigmoid) activation function. the latter seems to be simply (though usefully) a way of getting greater flexibility in the problems artificial neural networks can solve, and the former are examples of types of global behaviour that such networks can exhibit. it's a nifty concept, but i didn't really see what this chapter was trying to accomplish. (as a side nitpick, it also seemed to have been written by several people in conjunction, since they introduced some of the concepts more than once.)
but the idea that these behaviours can be modelled by relatively simple equations is kind of nifty (though i doubt the pieces fit together as neatly as elman et al try to argue). if, for a given problem, we can find equations that cleanly map the solutions found by both human studies and neural networks, then why not just hardwire those solutions into a computer? is there something more subtle going on than the specific solutions, or are neural networks just a way to help collect data and confirm the mathematical solutions to learning problems?
Holy lots of information, batman. These two chapters were completely full of all the stuff we've been talking about since the beginning of class, especially in terms of information localization in the brain - and the possible death of wernike's and broca's areas of the brain (for speech and text understanding). It was interesting to see that these areas were not the holy grails of brain understanding, and that they could apparently be bypassed without a thought in children with damage in those areas of the brain. Plasticity is an extraordinary thing, and the book's emphasis on it (it comes up chapter after chapter) shows how close the connectionists hold it to their hearts in terms of their understanding of the brain and learning. For one thing, it absolutely screams non-specific localization, although it does not rule out localization entirely. What that means is that although information can be coded into areas of the brain, these areas are not specific to the brain, and they can vary across individuals (although because of how the input is routed and how it is given, patterns do emerge between members of the same species, etc.)
Chapter 4 was a little more esoteric than chapter 5 - its focus on nonlinear, nonmonotonic and discontinuous patterns of change was necessary and even a little interesting, but I got bogged down in the math and began to skip over equations and just look at all the pretty graphs every once in a while. The book stated that instead of looking for a two-state hypothesis to fit data, we should look for a single system. This is a better use of our methods of understanding systems, but I'm not so sure that it might help me construct a model that would be of use - perhaps once such a function is found for research data, the modeler should consider useing this function or some variation thereof as an activation function? But this seems like aping the data instead of understanding it. Is there a way that we can go from mathematical models of understanding (vector algebra and the resulting functions) to computational models of understanding (neural networks)? This I think would be an interesting topic of discussion.
oh yeah, and how about those names of all the types of cells in the brain? pretty strange - pyramidal cells, basket cells, chandelier cells, double bouquet cells, etc. Do all these cells look like their names, or was some researcher on crack when s/he named them? And what does a double bouquet look like? Just scamming for discussion topics.
My primary reaction to the first chapter was, "Wow, that had a LOT of math, and not that much content behind it." I suppose the first 60%-70% of the chapter was SORT of necessary for the last little bit, but only if the person reading it had very little mathematical background, and I would think that the target audience for this book would not fall into that category. In addition, I think that they could have said something different each time the presented a slightly more complicate differential equation besides, "See, it follows behavior."
In any case, I did enjoy the last quarter of the chapter. I've never really seen the concept of an attractor before, and I was excited to see how well it appeared to explain some things. In Paul Churchland's book, "The Engine of Reason, The Seat of the Soul," he describes cyclical patterns in an activation space as an explanation for repetitive movements. If we think of a recurrent network set up correctly, as long as there is no external input, the pattern (e.g., walking) would just be repeated because the system was in its limit.
Also, even though the bifurication concept just served to drive home the point, which I would think was already pretty deep, of "nurture" vs. "Nature," I did think that it was an elegant explanation of the variety we see around us.
aahhh, 60+ pages of math...at least the first chapter was not that intense and allowed some skimming. but there was hardly any new information contained, my math profs have been good! the principle about the iterations has, however, never really occurred to me as such and it seems to be very interesting, especially when applied to neural nets, which sometimes need specific outputs such as 1.0 and 0.0, and such iterations allow values at points of attraction to happen.
i don't quite believe the 'readiness' of the network after exactly 17,999 sweeps and the instantaneous appreciation of the parity task with only one pattern, but I assume thatthese figures were rounded a bit although they do describe the real event of 1-step learning.
boy was i glad to meet the english past tense experiment yet again. same book, different chapter.
i have not been able to start the second chapter yet, but i hope to do so before 4 today. sorry about this...