Thread: Artificial Neural Networks

So it would be low-level, then?

As in, just a net of billions of, er, neurons, that exchange information like Memory adresses and when/ how to push bits around?

Because that's basically what our brains do.

Says who?

Nobody knows what corrosponds to a 'bit' in brain terms. It may well use some very exotic form of computation in which the term 'bit' is meaningless. To say how the brain does or doesn't work at the moment is very very premature.

Well, that would be as low level as you could possibly get in a computer, so it doesn't really matter, but don't Neurons use charges that are sent as Synapses? If it's just a charge, and then that is ended with some kind of reaction in the body, i.e. Putting your hand on something hot, the charge sent from the neurons in your hand cause the muscles to move as a knee-jerk response to the situation, not a conscious effort. Or like, when you move any part of your body, that would be your neurons stimulating your muscles into moving. Thinking is what we really don't know much about. How can you explain how thought occurs using non technical terms?

Originally Posted by A Roxxor

You mean like the Jabberwacky program that can link words and emotions together and respond semi intelligently?

Or something different?

That's one approach of many. Check out this tree.
http://www.forecastingprinciples.com...ologytree.html

It's a little confusing at first glance, but explains the different approaches quite well. I don't know the specifics of the Jabberwacky program, but you could clearly fit it into one of those categories.

Originally Posted by Xei

There's so many unfalsifiable or even meaningless models... there's just no way you can find out how the brain works via studying a tiny subset of its actions and reactions, and in an unthorough way.

And if I could point out; none of these people have so far made any progress in the field.

The only way to do it is to do it properly. It's exactly the same problem that was faced with the 'what is a gene and how does a gene work' issue. There's no way you can work it out by studying entire organisms. To explain it properly, you have to work out what the actual code is, and how it works. That has now been achieved, and the benefits to science have been huge.

Regarding unfalsifiable models: this is simply not the case. Certainly it was the case in the early years of psychology (I shudder to think of psychoanalytic and so-called humanistic theories ), but because of this fact, modern psychologists are particularly wary of repeating those old Freudian mistakes, and they go to great lengths to produce theories that derive clear-cut and testable predictions. A classic example from my own field of social psychology is that of cognitive dissonance theory. The full details of dissonance theory are outside the scope of this thread (there are many online resources if you're interested), but essentially it offers real and testable explanations that account for many cases of attitude formation and change. These predictions have been verified via a wide range of different research methods and the theory continues to be expanded upon to this day.

Regarding finding out how the brain works: this is really only a subset of the goals of psychology. Most psychologists take a more holistic approach to understanding human behavior and mental processes. In terms of levels of analysis, behavior may be understood in terms of low-level processes (i.e., the biological level), high-level processes (the representational level, e.g. attitudes, beliefs and other aspects of conscious experience), or the mid-level processes that mediate the two (the computational level, e.g. cognitive processes such as attention filtering). Ideally we wish to examine how we get from one to another and how they all function as a coherent system. In particular, we need more research tying together the biological and computational levels, which is essentially what neural network research (which, like it or not, falls under the realm of cognitive psychology, or at best the "cognitive science" meta-field I mentioned above which contains cognitive psychology) aims to do. We actually do know a fair amount about the biological level in isolation (in terms of neuronal functioning, electrochemical processes, brain structures, etc.), but these facts are largely meaningless without the higher levels of analysis to tie them together and make them relevant.

To understand the entire system, you examine not only the individual aspects of the system, but also how these components function together. As an analogy, we can learn a thing or two about computers from studying the hardware, but we learn a lot more about them from studying the operating systems and various forms of software (higher levels of analysis) and how the two levels interact with each other.

One final thing that I would like point out is that society has already derived very real (albeit often under-appreciated) benefits from psychological inquiry. I won't attempt to catalog them all (in no small part because even I don't fully appreciate the entire range of them), but as examples I will point to business applications such as marketing strategies and negotiating tactics, medical care and therapy (medical decision making, therapeutic techniques), various social and political policies, and legal/courtroom applications just to mention a handful - not to mention the potential benefits to everyday living that we get simply by having a greater understanding of how human behavior and mental processes really work.

Originally Posted by Xei

Did you say you were a psychologist, by the way? If so I'm not trying to be rude... just honest.

And if I'm just being ignorant perhaps you could clear some stuff up.

No offense taken. I am not, strictly speaking, a psychologist. I am wrapping up my undergraduate degree in psychology and am currently in the process of interviewing with several top-tier Ph.D. programs in social psychology. I intend to make a career as a professor and researcher in a sort of gray area of psychology between social, cognitive, and neuroscience. It is possible that a decade from now you and I will be reading each others' publications or even collaborating. (By the way, a Ph.D. is the entry-level degree in the field, so you should plan on going "all the way" if you intend to be a researcher.)

I'm lost on what this actually is... From what I can gather is that it helps predict stocks. Is it like AI? Can it forcast other things too?

Originally Posted by NeoSioType

I'm lost on what this actually is... From what I can gather is that it helps predict stocks. Is it like AI? Can it forcast other things too?

You could say it's a little like AI. A neural network is a system of neurons hooked to each other. The best example? A human brain. An artificial neural network is a model of a neural network typically used in math or computer science. It is most commonly thought of as a method of computing, one that we do not yet fully understand. You can apply it to lots of stuff, but it is best suited to a situation where it will be searching for patterns(like finding patterns in the stock market) or actually learning by creating rules for decision making(like a scalable AI).

Neural nets are basically systems that are capable of learning. Their most important feature is the ability to identify and recognize patterns. If there is a pattern present in the data, no matter how subtle or abstract, neural nets are capable of finding and using those patterns.

Say you need a program that can recognize apples when presented with a random fruit. To do that with a traditional program, just inturpreting the visual data into something you can work with is a daunting task. But with a neural net designed to output a yes or no to the apple question, you just tell the network wether it was wrong or right, in essence training it to recognize the characteristics of an apple. The network holds every example of an apple it has seen, and is able to extrapolate what they all have in common to corretly identify apples from random fruit.

Neural nets are commonly use in predicting stock market behavior in the hopes that it will uncover some pattern that will give the investor an advantage.

for some great examples of neural networks, I recommend http://www.20q.net/ and http://www.cleverbot.com/

Well, the perfect neural network would simply be a completely accurate model of our brains. That's what is being aimed for. There's no reason to doubt their abilities, unless you think there's something involved in brain activity other than neurons, or maybe if you hold some kind of weird anti-functionalist philosophical belief.

Don't you mean anti-reductionist? Because the functionalist theory of mental representation is exactly that there's something involved in brain activity other than neurons.

My basic understanding of simulations of neural nets is that they're currently very limited by the need to reinforce the result that is desired in an almost arbitrary way, but I haven't looked into the subject in any depth. From what I understand, very basic artificial models work on kind of 'trip' switches that act as 'neurons' and which learn through each switch having a 'weighting' that determines the chance of it turning on given an input. Weighting therefore acts as the learning process by helping to reinforce pathways through the neural net. But then again, this is coming from my hazy recollection of a lecture two years ago.

Regarding the debate about psychology, I think psychology is a very important tool indeed. Much of psychology is very important, but is limited by the black-box scenario: we don't properly understand the brain, and so psychology is the attempt to learn about it by controlling input and observing output, without actually being able to see what happens.

It's no different to the development of other sciences: theory advances before the technology or the means to test the theory. Once technology advances to the stage of being able to properly test these theories, they quickly become out of date, but without them there would be no working hypotheses to test at all. Freud's psychoanalytic theory of the mind seems hugely laughable now, but at the time it was revolutionary, and without it psychology would not have developed. Already we're seeing that much of philosophy of mind and psychology is being refuted through the actual ability to test these theories properly. The result is cognitive neuroscience.

I think that some of the major changes that will happen as we understand more about the brain is that the analogies between brain and computer will begin to break down. I'm coming at all of this from a philosophy of mind perspective, and one of the most important developments in the field was the emergence of computers. Through the eighties and nineties, philosophy of mind began to draw upon ideas of software and modularity to form analogies and metaphors for theories of mind. They helped to form ways of thinking about the brain, but I think that recent developments are starting to show that these analogies are very limited.

As with any emerging field in science, the first challenge is working out how to think about the problem. Then it becomes a matter of working out how to test those ideas, then how to put the resulting knowledge to good use.

The 'mystery' of the brain doesn't help, either. We actually know a surprising amount about the physical functioning of neurons. We know how they pass chemical messages on to other neurons, and how they connect in nets. Because of the complexity of the brain, though, there is a tendency to become anti-reductionist and feel that there must be 'something else' other than highly specialised nerve cells firing when we feel joy while looking at a piece of art.

btw, I don't quite see how a neural network is going to help in predicting stock market fluctuations. I can see how the markets can be compared to neural networks, and how this maybe fits into other ideas such as network theory, but forecasting? Oh well.

Hmm, my understanding of functionalism is the idea that the algorithm enacted inside the brain is what gives rise to the mind.

In The Emperor's New Mind, that's the term that Penrose used extensively. It's what he was trying to argue against; he was trying to argue that you can't explain the brain in terms of an algorithm, ie. that functionalism is wrong.

Not that I agreed with him. The simple fact is that such a stance asserts that there is something especially conscious about organic elements and water and ions etcetera, which I find untenable.

Functionalism has been around in philosophy of mind for a while now, at least since mid-seventies. There's a few different forms of it, and I must admit that I've never heard of it in relation to algorithms, though.

Basically, it's trying to answer the basic question 'what does it take for mental state x to represent y,' or, 'what are mental states?'

Behaviourists thought that mental states were can be analysed by examining the behaviours they produced. So the mental state 'being thirsty' represented events by producing the action of 'searching for water' or some such thing. There were many problems with these conceptions of mind, however.

Functionalism developed out of the first ideas of computing: it's very closely tied with the Turing machine. Functionalism allows for interaction between mental states, or mental-to-mental causation. The mental state 'pain,' for example represents the state 'being in pain' just in case of all the similar states it produces, for example the state 'i should take my hand off that hot plate,' or 'what the hell is that?' etc etc. A state represents by the function that the state has when put in the context of all other states.

Initially functionalism was taken to reducible to physical states, ie. mental state = physical state, but there was a problem of multiple realizability. If we take a universal state, such as pain, we can see that it is realized in many different species with remarkably different physical make-ups: pain is found in humans and octopi, for example. Do we then say that these are different mental states? If so, we would have to create species-indexical definitions of states (octopus-pain, bird-pain) and we would even have to narrow the definition further to individual organisms. We would then lose the unifying idea of having the same 'mental state' when in pain.

The solution to this was non-reductive physicalism for functionalists. Functionalists thought that, while there was nothing 'else' when it came to mental representation other than the physical substance (in other words, they were avoiding dualism for obvious reasons), mental states could not be reduced purely to physical states. There's some pretty big problems with this, and I think now nonreductive functionalism has had its day.

So there's a little introduction to the philosophy of mind

Maybe Penrose was simply using 'algorithm' to refer to the functional state.

If you take a functionalist view of the mind, just a pattern within a system of like elements, then you have to consider that intelligence could arise anywhere.

Like a flock of bird all flying perfectly together, each bird represents a neuron, and their interacting give rise to a sort of group consciousness.

It could be the interaction of leaves blowing in the wind, or water molecules. It could even happen with bicycle parts, if you built a machine out of bike parts about the size of Jupiter.

But there always seems to be something missing from that approach, some essential element, a spark of life or consciousness. To me it seems the missing ingredient is displayed in computer viruses, and if you could cross a virus with an AI, we'd have robot overlords in no time!

Yes but the key word is 'could'. I take the functionalist view but I realise that it's likely that a special kind of pattern must emerge. The field is far to new to be able to answer what though. Is a flock of birds conscious? It's not really possible to tell. If 100 billion clowns were all to throw pies at each other in a way which flawlessly emulated neural activity, would it be conscious? I've never heard a good reason why not.

If you take a functionalist view of the mind, just a pattern within a system of like elements, then you have to consider that intelligence could arise anywhere.

Like a flock of bird all flying perfectly together, each bird represents a neuron, and their interacting give rise to a sort of group consciousness.

Sort of. A key tenet of Functionalism is precisely Multiple Realizability (the ability for multiple physically distinct systems to realize a mental state).The most common comparison in philosophy of mind literature is (bizzarely) octopi and humans.

Octopi are different enough from humans that it is safe to believe their brains work significantly differently. Now, if we were to have a device to pinch an octopus' tentacle, and we see it recoil the tentacle in shock, we would usually say that it is feeling the mental state of 'pain.' We obviously don't have access to the octopus' intensional (with an s) state, so we cannot be sure, but from observing the octopus' behaviour we can infer it pretty readily.

How do we do so? We observe that the physical stimulus of the pinch (call it P*) leads to sudden evasive action (E) and perhaps cautious behaviour when approaching the pinching-device in the future/unwillingness to approach the device. The thing about functionalism is that it doesn't necessarily depend on any behaviours, but on inter-related mental states. So, pain is defined as something like:

Octopi are in pain just in case: physical stimuli P* leads to mental state E (evasive action) and results in C (future cautious behaviour). Whenever these states (or others - 'crying' or 'rush of adrenalin' for example) are instantiated, whatever that mental state is, we call it 'pain'.

Obviously there are many more complex states involved in feeling pain, such as perhaps 'seeking medical help' etc, but this is a general example. The mental states depend upon the function they perform via linking to other mental states, and the behaviours/actions these cause.

There are a few problems with functionalism, the primary one being a problem constructed by Jaegwon Kim against non-reductive physicalism (which is what functionalism is), which concerns mental causation. I'm happy to outline it if anyone is interested.