Artificial Neural Networks

[The Cusp]

Anyone at all familiar with neural networks? It looks like there is various software available which allows you to fool around with your own. Kind of like your own Pokemon, you train it to do certain things based on certain input of stimuli.

Stuff like predicting the stock market with whatever factors you want to make relevant, like oil or gas prices, market changes, the scores sports games, anything at all. You could even use online dream journal entries.

The beauty about neural nets is the find subtle relations between all those inputs in ways we would never think of. There was one experiment done with a neural net hooked up to a circuit board full of various transistors, capacitors, and various electrical doo dads. The would train it to do various tasks using the components it knew nothing about. It accomplished one of those tasks in a manner that still has electrical engineers baffled to this day, because they can't figure out how it works. According to science, that circuit shouldn't be doing what it does, but it somehow works, and nobody can figure out why or how. Gotta be a web link to that somewhere, but I can't find one at the moment. I have it in a book.

So anyone know anything about setting up neural networks? I haven't really looked at any of the option in detail yet.

This site looks promising. http://www.neural-forecasting.com/

[Xei]

I'd like to see where you can get a cool program where you can mess around with them easily. Know of any?

This is actually what I want to research as a career. In my opinion it's at the top of the list of the greatest remaining scientific mysteries... there is currently pretty much no understanding at all of the very basics of how the neurons of the cerebral cortex work.

[The Cusp]

There is a free downloadable CD here with everything you need to get started.
http://www.neural-forecasting.com/free_software_cd.htm

[metcalfracing]

This is interesting... I'm checking it out

[metcalfracing]

haha, only like 1 or 2 of those programs run on 64bit vista...

[Xei]

What's neural forecasting?

The only result for it on Google is that site, and that site doesn't explain it.

[The Cusp]

What's neural forecasting?

The only result for it on Google is that site, and that site doesn't explain it.

Basically using a neural network to predict something like stock market behavior.

[Xei]

Bleh, I'm not particularly interested in that.

I'm more interested in how actual cognitive functions work, which nobody has figured out yet.

[12g951ad]

hmm new age crap anyone ?.. . . ?

[Xei]

Yeah you'll find all your new age crap related needs here:

http://www.dreamviews.com/community/forumdisplay.php?f=19

Bye bye.

[12g951ad]

Thanks !
but not the point

[Xei]

You don't have a point.

Do you not understand some of the big words that are being used here or something? Neural networks are a field of serious scientific study, there is nothing 'new age' about them.

[metcalfracing]

hmm new age crap anyone ?.. . . ?

I don't even understand how you can draw a comparison between neural networking and new age hippy crap...

Neural networks are just computational models used to drawing comparisons from data... among other things

[DuB]

It's certainly an interesting area of research. There are some who think that theories of cognition are increasingly going to based on computational models. I'm somewhat skeptical, however, as developing these models requires a large amount of training in mathematics and, to a lesser degree, computer programming, and the people who have traditionally addressed these research questions (i.e., cognitive psychologists) tend to receive their mathematical training in statistics and quantitative measurement.

The emerging, interdisciplinary field of "cognitive science" seems promising. It will be interesting to see how they approach these questions.

[Xei]

I don't see how else we can figure out how the mind works properly if we don't even know how neural networks work.

To be frank, the current knowledge about neuroscience is unimpressive. We know that the brain's made of neurons. That's about it.

'Cognitive psychology' or whatever are never going to get us anywhere. The only way forwards is a brute force cracking of how the brain actually works.

Otherwise it's like trying to design a computer or figure out how one works with only the knowledge that 'it's made of transistors'.

[DuB]

Compared to more established sciences, I agree that it is somewhat disappointing to think of how little neuroscience has really given us. I view it as one of the last frontiers of scientific investigation. The relatively recent development of functional brain imaging has opened up a lot of possibilities, and I am hopeful that soon we can begin tackling the bigger questions.

I have to say that I am a little confused at your comment, "we don't even know how neural networks work." The entire point of neural networks (at least the way that I understand it - which I admit may be flawed) is that we do understand how they work, and that this fact allows us to learn a thing or two about how the cognitive processes that they emulate work. Whereas we cannot break open the black box that is the human mind and examine how it is functioning (at least not currently, or probably any time soon), we can do just that with neural networks and other computational models. So if we have a computational model that appears to mimic human cognition, we can infer that they must be utilizing similar processes. Is this not the way that you understand neural networks?

On a side note: I find the analogy of using "brute force" to understand brain functioning to be very amusing. I have never viewed science as a "brute force" sort of process, but to each his own, I guess. :lol:

[Xei]

Well, psychology just strikes me as being science for pussies. :P

To understand the thing properly you've gotta take it down to the fundamental level...

And about neural networks; nah, it isn't understood how they perform computations etcetera. With computers we know that it uses binary code to represent numbers, etcetera; we don't know what the 'code' is for the brain. I remember the head of the Blue Brain project saying that the field is waiting for a conceptual breakthrough. Until then it's just groping in the dark...

It's something I plan to help crack, once I've got a degree in maths and maybe a master's in neuroscience. :)

[DuB]

Well, psychology just strikes me as being science for pussies. :PHeh. Okay, I'll bite... :wink:

What exactly is it about psychology that strikes you as being, ahem, "for pussies?" (Ironically, many of the people currently studying artificial neural networks received their Ph.D.s in cognitive psychology.)

In point of fact, science is a method rather than a predefined set of topics or body of knowledge, and anything can be studied via the scientific method - yes, even people's thoughts, feelings, and behavior. I've found that people who accuse psychology of lacking rigor tend to misunderstand how it is studied. It doesn't help that psychologists are routinely portrayed by the media as headshrinkers who analyze your dreams while you lie facing away from them on a couch doing free associations :roll:. Suffice it to say that this is not an accurate view of what psychology is all about. (I wonder if wearing white lab coats and carrying around colorful beakers could improve our reputation? :chuckle:)

Certainly psychology faces a unique set of challenges by studying what is probably the most complex and dynamic thing on the planet, namely the inner life of human beings. I could rather easily make the case that studying things as relatively simple as elements and atoms is "science for pussies" (but I won't ;)). Rather than becoming dismayed at the difficulty of scientifically studying people's thoughts, feelings, and behavior, I say that we delve in and try to sort things out as best we can. Psychology is a young science and there is so much left to learn - I find it to be very exciting.

[A Roxxor]

Have Neural Networks ever been observed to become cognitive in any way?

[Xei]

Yeah, it's called a brain.
What exactly is it about psychology that strikes you as being, ahem, "for pussies?" (Ironically, many of the people currently studying artificial neural networks received their Ph.D.s in cognitive psychology.)
Everybody knows it isn't a 'proper' science. There's so many unfalsifiable or even meaningless models... there's just no way you can find out how the brain works via studying a tiny subset of its actions and reactions, and in an unthorough way.

And if I could point out; none of these people have so far made any progress in the field.

The only way to do it is to do it properly. It's exactly the same problem that was faced with the 'what is a gene and how does a gene work' issue. There's no way you can work it out by studying entire organisms. To explain it properly, you have to work out what the actual code is, and how it works. That has now been achieved, and the benefits to science have been huge.

Did you say you were a psychologist, by the way? If so I'm not trying to be rude... just honest. :P

And if I'm just being ignorant perhaps you could clear some stuff up.

[A Roxxor]

Whoops, I meant Artificial Neural Networks...

[The Cusp]

'Cognitive psychology' or whatever are never going to get us anywhere. The only way forwards is a brute force cracking of how the brain actually works.

That's exactly what artificial neural networks allow us to do.

[Xei]

Yeah, I know. Although they still haven't actually figured out how they work yet.
Whoops, I meant Artificial Neural Networks...
No, not yet. In the same way it was many years before uranium 235 was observed to undergo stable fission. They're working on it.

[The Cusp]

Have Neural Networks ever been observed to become cognitive in any way?

The short answer is no. But... There are a lot of experiments where it kind of, sort of becomes cognitive in some way. All very technical, but that's where it get interesting and raises all sorts of philosophical questions.

Plus there's all kinds of gray in between ground. Stuff like biological computers, mice with artificial neural nets connected to their brains. Never anything concrete, but the potential is always there, teasing.

[A Roxxor]

You mean like the Jabberwacky program that can link words and emotions together and respond semi intelligently?

Or something different?

Neural Nets are so abstract...

[Xei]

Well, the perfect neural network would simply be a completely accurate model of our brains. That's what is being aimed for. There's no reason to doubt their abilities, unless you think there's something involved in brain activity other than neurons, or maybe if you hold some kind of weird anti-functionalist philosophical belief.

[A Roxxor]

So it would be low-level, then? :?

As in, just a net of billions of, er, neurons, that exchange information like Memory adresses and when/ how to push bits around?

Because that's basically what our brains do.

[Xei]

Says who?

Nobody knows what corrosponds to a 'bit' in brain terms. It may well use some very exotic form of computation in which the term 'bit' is meaningless. To say how the brain does or doesn't work at the moment is very very premature.

[A Roxxor]

Well, that would be as low level as you could possibly get in a computer, so it doesn't really matter, but don't Neurons use charges that are sent as Synapses? If it's just a charge, and then that is ended with some kind of reaction in the body, i.e. Putting your hand on something hot, the charge sent from the neurons in your hand cause the muscles to move as a knee-jerk response to the situation, not a conscious effort. Or like, when you move any part of your body, that would be your neurons stimulating your muscles into moving. Thinking is what we really don't know much about. How can you explain how thought occurs using non technical terms?

[The Cusp]

You mean like the Jabberwacky program that can link words and emotions together and respond semi intelligently?

Or something different?

That's one approach of many. Check out this tree.
http://www.forecastingprinciples.com/methodologytree.html

It's a little confusing at first glance, but explains the different approaches quite well. I don't know the specifics of the Jabberwacky program, but you could clearly fit it into one of those categories.

[DuB]

There's so many unfalsifiable or even meaningless models... there's just no way you can find out how the brain works via studying a tiny subset of its actions and reactions, and in an unthorough way.

And if I could point out; none of these people have so far made any progress in the field.

The only way to do it is to do it properly. It's exactly the same problem that was faced with the 'what is a gene and how does a gene work' issue. There's no way you can work it out by studying entire organisms. To explain it properly, you have to work out what the actual code is, and how it works. That has now been achieved, and the benefits to science have been huge.Regarding unfalsifiable models: this is simply not the case. Certainly it was the case in the early years of psychology (I shudder to think of psychoanalytic and so-called humanistic theories :roll:), but because of this fact, modern psychologists are particularly wary of repeating those old Freudian mistakes, and they go to great lengths to produce theories that derive clear-cut and testable predictions. A classic example from my own field of social psychology is that of cognitive dissonance theory. The full details of dissonance theory are outside the scope of this thread (there are many online resources if you're interested), but essentially it offers real and testable explanations that account for many cases of attitude formation and change. These predictions have been verified via a wide range of different research methods and the theory continues to be expanded upon to this day.

Regarding finding out how the brain works: this is really only a subset of the goals of psychology. Most psychologists take a more holistic approach to understanding human behavior and mental processes. In terms of levels of analysis, behavior may be understood in terms of low-level processes (i.e., the biological level), high-level processes (the representational level, e.g. attitudes, beliefs and other aspects of conscious experience), or the mid-level processes that mediate the two (the computational level, e.g. cognitive processes such as attention filtering). Ideally we wish to examine how we get from one to another and how they all function as a coherent system. In particular, we need more research tying together the biological and computational levels, which is essentially what neural network research (which, like it or not, falls under the realm of cognitive psychology, or at best the "cognitive science" meta-field I mentioned above which contains cognitive psychology) aims to do. We actually do know a fair amount about the biological level in isolation (in terms of neuronal functioning, electrochemical processes, brain structures, etc.), but these facts are largely meaningless without the higher levels of analysis to tie them together and make them relevant.

To understand the entire system, you examine not only the individual aspects of the system, but also how these components function together. As an analogy, we can learn a thing or two about computers from studying the hardware, but we learn a lot more about them from studying the operating systems and various forms of software (higher levels of analysis) and how the two levels interact with each other.

One final thing that I would like point out is that society has already derived very real (albeit often under-appreciated) benefits from psychological inquiry. I won't attempt to catalog them all (in no small part because even I don't fully appreciate the entire range of them), but as examples I will point to business applications such as marketing strategies and negotiating tactics, medical care and therapy (medical decision making, therapeutic techniques), various social and political policies, and legal/courtroom applications just to mention a handful - not to mention the potential benefits to everyday living that we get simply by having a greater understanding of how human behavior and mental processes really work.

Did you say you were a psychologist, by the way? If so I'm not trying to be rude... just honest. :P

And if I'm just being ignorant perhaps you could clear some stuff up.No offense taken. I am not, strictly speaking, a psychologist. I am wrapping up my undergraduate degree in psychology and am currently in the process of interviewing with several top-tier Ph.D. programs in social psychology. I intend to make a career as a professor and researcher in a sort of gray area of psychology between social, cognitive, and neuroscience. It is possible that a decade from now you and I will be reading each others' publications or even collaborating. (By the way, a Ph.D. is the entry-level degree in the field, so you should plan on going "all the way" if you intend to be a researcher.)

[NeoSioType]

I'm lost on what this actually is... From what I can gather is that it helps predict stocks. Is it like AI? Can it forcast other things too?

[Licity]

I'm lost on what this actually is... From what I can gather is that it helps predict stocks. Is it like AI? Can it forcast other things too?

You could say it's a little like AI. A neural network is a system of neurons hooked to each other. The best example? A human brain. An artificial neural network is a model of a neural network typically used in math or computer science. It is most commonly thought of as a method of computing, one that we do not yet fully understand. You can apply it to lots of stuff, but it is best suited to a situation where it will be searching for patterns(like finding patterns in the stock market) or actually learning by creating rules for decision making(like a scalable AI).

[The Cusp]

Neural nets are basically systems that are capable of learning. Their most important feature is the ability to identify and recognize patterns. If there is a pattern present in the data, no matter how subtle or abstract, neural nets are capable of finding and using those patterns.

Say you need a program that can recognize apples when presented with a random fruit. To do that with a traditional program, just inturpreting the visual data into something you can work with is a daunting task. But with a neural net designed to output a yes or no to the apple question, you just tell the network wether it was wrong or right, in essence training it to recognize the characteristics of an apple. The network holds every example of an apple it has seen, and is able to extrapolate what they all have in common to corretly identify apples from random fruit.

Neural nets are commonly use in predicting stock market behavior in the hopes that it will uncover some pattern that will give the investor an advantage.

[metcalfracing]

for some great examples of neural networks, I recommend http://www.20q.net/ and http://www.cleverbot.com/

[Roller]

Well, the perfect neural network would simply be a completely accurate model of our brains. That's what is being aimed for. There's no reason to doubt their abilities, unless you think there's something involved in brain activity other than neurons, or maybe if you hold some kind of weird anti-functionalist philosophical belief.

Don't you mean anti-reductionist? Because the functionalist theory of mental representation is exactly that there's something involved in brain activity other than neurons.

My basic understanding of simulations of neural nets is that they're currently very limited by the need to reinforce the result that is desired in an almost arbitrary way, but I haven't looked into the subject in any depth. From what I understand, very basic artificial models work on kind of 'trip' switches that act as 'neurons' and which learn through each switch having a 'weighting' that determines the chance of it turning on given an input. Weighting therefore acts as the learning process by helping to reinforce pathways through the neural net. But then again, this is coming from my hazy recollection of a lecture two years ago.

Regarding the debate about psychology, I think psychology is a very important tool indeed. Much of psychology is very important, but is limited by the black-box scenario: we don't properly understand the brain, and so psychology is the attempt to learn about it by controlling input and observing output, without actually being able to see what happens.

It's no different to the development of other sciences: theory advances before the technology or the means to test the theory. Once technology advances to the stage of being able to properly test these theories, they quickly become out of date, but without them there would be no working hypotheses to test at all. Freud's psychoanalytic theory of the mind seems hugely laughable now, but at the time it was revolutionary, and without it psychology would not have developed. Already we're seeing that much of philosophy of mind and psychology is being refuted through the actual ability to test these theories properly. The result is cognitive neuroscience.

I think that some of the major changes that will happen as we understand more about the brain is that the analogies between brain and computer will begin to break down. I'm coming at all of this from a philosophy of mind perspective, and one of the most important developments in the field was the emergence of computers. Through the eighties and nineties, philosophy of mind began to draw upon ideas of software and modularity to form analogies and metaphors for theories of mind. They helped to form ways of thinking about the brain, but I think that recent developments are starting to show that these analogies are very limited.

As with any emerging field in science, the first challenge is working out how to think about the problem. Then it becomes a matter of working out how to test those ideas, then how to put the resulting knowledge to good use.

The 'mystery' of the brain doesn't help, either. We actually know a surprising amount about the physical functioning of neurons. We know how they pass chemical messages on to other neurons, and how they connect in nets. Because of the complexity of the brain, though, there is a tendency to become anti-reductionist and feel that there must be 'something else' other than highly specialised nerve cells firing when we feel joy while looking at a piece of art.

btw, I don't quite see how a neural network is going to help in predicting stock market fluctuations. I can see how the markets can be compared to neural networks, and how this maybe fits into other ideas such as network theory, but forecasting? Oh well.

[Xei]

Hmm, my understanding of functionalism is the idea that the algorithm enacted inside the brain is what gives rise to the mind.

In The Emperor's New Mind, that's the term that Penrose used extensively. It's what he was trying to argue against; he was trying to argue that you can't explain the brain in terms of an algorithm, ie. that functionalism is wrong.

Not that I agreed with him. The simple fact is that such a stance asserts that there is something especially conscious about organic elements and water and ions etcetera, which I find untenable.

[Roller]

Functionalism has been around in philosophy of mind for a while now, at least since mid-seventies. There's a few different forms of it, and I must admit that I've never heard of it in relation to algorithms, though.

Basically, it's trying to answer the basic question 'what does it take for mental state x to represent y,' or, 'what are mental states?'

Behaviourists thought that mental states were can be analysed by examining the behaviours they produced. So the mental state 'being thirsty' represented events by producing the action of 'searching for water' or some such thing. There were many problems with these conceptions of mind, however.

Functionalism developed out of the first ideas of computing: it's very closely tied with the Turing machine. Functionalism allows for interaction between mental states, or mental-to-mental causation. The mental state 'pain,' for example represents the state 'being in pain' just in case of all the similar states it produces, for example the state 'i should take my hand off that hot plate,' or 'what the hell is that?' etc etc. A state represents by the function that the state has when put in the context of all other states.

Initially functionalism was taken to reducible to physical states, ie. mental state = physical state, but there was a problem of multiple realizability. If we take a universal state, such as pain, we can see that it is realized in many different species with remarkably different physical make-ups: pain is found in humans and octopi, for example. Do we then say that these are different mental states? If so, we would have to create species-indexical definitions of states (octopus-pain, bird-pain) and we would even have to narrow the definition further to individual organisms. We would then lose the unifying idea of having the same 'mental state' when in pain.

The solution to this was non-reductive physicalism for functionalists. Functionalists thought that, while there was nothing 'else' when it came to mental representation other than the physical substance (in other words, they were avoiding dualism for obvious reasons), mental states could not be reduced purely to physical states. There's some pretty big problems with this, and I think now nonreductive functionalism has had its day.

So there's a little introduction to the philosophy of mind :)

Maybe Penrose was simply using 'algorithm' to refer to the functional state.

[The Cusp]

If you take a functionalist view of the mind, just a pattern within a system of like elements, then you have to consider that intelligence could arise anywhere.

Like a flock of bird all flying perfectly together, each bird represents a neuron, and their interacting give rise to a sort of group consciousness.

It could be the interaction of leaves blowing in the wind, or water molecules. It could even happen with bicycle parts, if you built a machine out of bike parts about the size of Jupiter.

But there always seems to be something missing from that approach, some essential element, a spark of life or consciousness. To me it seems the missing ingredient is displayed in computer viruses, and if you could cross a virus with an AI, we'd have robot overlords in no time!

[Xei]

Yes but the key word is 'could'. I take the functionalist view but I realise that it's likely that a special kind of pattern must emerge. The field is far to new to be able to answer what though. Is a flock of birds conscious? It's not really possible to tell. If 100 billion clowns were all to throw pies at each other in a way which flawlessly emulated neural activity, would it be conscious? I've never heard a good reason why not.

[Roller]

If you take a functionalist view of the mind, just a pattern within a system of like elements, then you have to consider that intelligence could arise anywhere.

Like a flock of bird all flying perfectly together, each bird represents a neuron, and their interacting give rise to a sort of group consciousness.

Sort of. A key tenet of Functionalism is precisely Multiple Realizability (the ability for multiple physically distinct systems to realize a mental state).The most common comparison in philosophy of mind literature is (bizzarely) octopi and humans.

Octopi are different enough from humans that it is safe to believe their brains work significantly differently. Now, if we were to have a device to pinch an octopus' tentacle, and we see it recoil the tentacle in shock, we would usually say that it is feeling the mental state of 'pain.' We obviously don't have access to the octopus' intensional (with an s) state, so we cannot be sure, but from observing the octopus' behaviour we can infer it pretty readily.

How do we do so? We observe that the physical stimulus of the pinch (call it P*) leads to sudden evasive action (E) and perhaps cautious behaviour when approaching the pinching-device in the future/unwillingness to approach the device. The thing about functionalism is that it doesn't necessarily depend on any behaviours, but on inter-related mental states. So, pain is defined as something like:

Octopi are in pain just in case: physical stimuli P* leads to mental state E (evasive action) and results in C (future cautious behaviour). Whenever these states (or others - 'crying' or 'rush of adrenalin' for example) are instantiated, whatever that mental state is, we call it 'pain'.

Obviously there are many more complex states involved in feeling pain, such as perhaps 'seeking medical help' etc, but this is a general example. The mental states depend upon the function they perform via linking to other mental states, and the behaviours/actions these cause.

There are a few problems with functionalism, the primary one being a problem constructed by Jaegwon Kim against non-reductive physicalism (which is what functionalism is), which concerns mental causation. I'm happy to outline it if anyone is interested.