NeuroSolutions Frequently Asked Questions
(FAQ)Questions about the Technology
What is a
neural network?
What kind
of real-world problems can neural networks solve?
What are
genetic algorithms?
How can
genetic algorithms be used to improve neural networks?
Questions about the Technology in NeuroSolutions
What are
some of the types of neural networks that I can build with
NeuroSolutions?
What if
I know nothing about the various types of neural networks?
What
if the neural network type I am interested in is not listed?
Does
NeuroSolutions work with Excel files?
Once I
develop a working neural network in NeuroSolutions, how do I use
it?
What
Operating Systems does NeuroSolutions run on?
Q.
What is a Neural Network?
A neural network is a powerful data
modeling tool that is able to capture and represent complex input/output
relationships. The motivation for the development of neural network
technology stemmed from the desire to develop an artificial system that
could perform "intelligent" tasks similar to those performed by the human
brain. Neural networks resemble the human brain in the following two
ways:
- A neural network acquires knowledge through learning.
- A neural network's knowledge is stored within inter-neuron
connection strengths known as synaptic weights.
The true power and advantage of neural networks lies in their ability
to represent both linear and non-linear relationships and in their ability
to learn these relationships directly from the data being modeled.
Traditional linear models are simply inadequate when it comes to modeling
data that contains non-linear characteristics.
The most common neural network model is the multilayer perceptron
(MLP). This type of neural network is known as a supervised network
because it requires a desired output in order to learn. The goal of this
type of network is to create a model that correctly maps the input to the
output using historical data so that the model can then be used to produce
the output when the desired output is unknown. A graphical representation
of an MLP is shown below.
Block diagram
of a two hidden layer multiplayer perceptron (MLP). The inputs are fed
into the input layer and get multiplied by interconnection weights as they
are passed from the input layer to the first hidden layer. Within the
first hidden layer, they get summed then processed by a nonlinear function
(usually the hyperbolic tangent). As the processed data leaves the first
hidden layer, again it gets multiplied by interconnection weights, then
summed and processed by the second hidden layer. Finally the data is
multiplied by interconnection weights then processed one last time within
the output layer to produce the neural network output.
The MLP and many other neural
networks learn using an algorithm called backpropagation. With
backpropagation, the input data is repeatedly presented to the neural
network. With each presentation the output of the neural network is
compared to the desired output and an error is computed. This error is
then fed back (backpropagated) to the neural network and used to adjust
the weights such that the error decreases with each iteration and the
neural model gets closer and closer to producing the desired output. This
process is known as "training".
Demonstration
of a neural network learning to model the exclusive-or (Xor) data. The Xor
data is repeatedly presented to the neural network. With each
presentation, the error between the network output and the desired output
is computed and fed back to the neural network. The neural network uses
this error to adjust its weights such that the error will be decreased.
This sequence of events is usually repeated until an acceptable error has
been reached or until the network no longer appears to be learning.
A good way to introduce the topic is to take a look at a typical
application of neural networks. Many of today's document scanners for the
PC come with software that performs a task known as optical character
recognition (OCR). OCR software allows you to scan in a printed document
and then convert the scanned image into to an electronic text format such
as a Word document, enabling you to manipulate the text. In order to
perform this conversion the software must analyze each group of pixels
(0's and 1's) that form a letter and produce a value that corresponds to
that letter. Some of the OCR software on the market use a neural network
as the classification engine.
Demonstration
of a neural network used within an optical character recognition (OCR)
application. The original document is scanned into the computer and saved
as an image. The OCR software breaks the image into sub-images, each
containing a single character. The sub-images are then translated from an
image format into a binary format, where each 0 and 1 represents an
individual pixel of the sub-image. The binary data is then fed into a
neural network that has been trained to make the association between the
character image data and a numeric value that corresponds to the
character. The output from the neural network is then translated into
ASCII text and saved as a file.
 Return to
Topic List
Q. What kind of real-world problems can neural
networks solve?
A. Neural networks can be applied to almost any
problem where you have 1) historical data and 2) a need to create a model
for that data. Neural networks have been successfully applied to broad
spectrum of data-intensive applications, such as:
- Process Modeling and Control - Creating a neural network
model for a physical plant then using that model to determine the best
control settings for the plant.
- Machine Diagnostics - Detect when a machine has failed so
that the system can automatically shut down the machine when this
occurs.
- Portfolio Management - Allocate the assets in a portfolio in
a way that maximizes return and minimizes risk.
- Target Recognition - Military application which uses video
and/or infrared image data to determine if an enemy target is present.
- Medical Diagnosis - Assisting doctors with their diagnosis by
analyzing the reported symptoms and/or image data such as MRIs or
X-rays.
- Credit Rating - Automatically assigning a company's or
individuals credit rating based on their financial condition.
- Targeted Marketing - Finding the set of demographics which
have the highest response rate for a particular marketing campaign.
- Voice Recognition - Transcribing spoken words into ASCII
text.
- Financial Forecasting - Using the historical data of a
security to predict the future movement of that security.
- Quality Control - Attaching a camera or sensor to the end of
a production process to automatically inspect for defects.
- Intelligent Searching - An internet search engine that
provides the most relevant content and banner ads based on the users'
past behavior.
- Fraud Detection - Detect fraudulent credit card transactions
and automatically decline the charge.
- Optical Character Recognition (OCR) - Example shown above
where a printed document is scanned and converted to an electronic
format such as ASCII text, which can then be manipulated and/or stored
more efficiently.
Return to
Topic List
Q.
What are Genetic
Algorithms?
A. Genetic algorithms are general-purpose search
algorithms based upon the principles of evolution observed in nature. Even
with today's high-powered computers, using an exhaustive search to find
the optimal solution for even relatively small problems can be
prohibitively expensive.
For many problems, genetic algorithms can often find good solutions
(near-optimal) in around 100 generations. This can be many times faster
than an exhaustive search. For example, a chromosome containing 32 binary
genes would have 4,294,967,296 possible combinations (solutions) to
evaluate when using an exhaustive search. If the same problem were to be
solved with a genetic algorithm of population size 50, requiring 100
generations of evolution, the genetic algorithm would only need to
evaluate 5000 possible solutions.
As mentioned previously, genetic algorithms are able to find optimal or
near optimal solutions by using many of the principles of evolution that
can be observed in nature. These include selection, crossover, and
mutation. Below is an outline of how a genetic algorithm uses these
principals to search for optimal solutions:
- Create an initial population of chromosomes. Each chromosome is made
up of a collection of genes (the parameters to be optimized) and
represents a complete solution to the problem at hand. The gene values
are usually initialized to random values within user-specified
boundaries.
- Evaluate each of the chromosomes in the initial population. A
chromosome is evaluated by a fitness function to determine the quality
of the solution. This fitness function is problem-specific and defines
the genetic algorithm's objective for the current problem.
- Select chromosomes that will have their information passed on to the
next generation. The most common selection operator is 뱑oulette
selection? This selection operator is based upon the evolutionary
principle know as 뱒urvival of the fittest? whereby a chromosome's
probability of getting selected is proportional to its fitness. Thus,
the good solutions are more likely to survive.
- Crossover the selected chromosomes to produce new offspring
chromosomes. This crossover occurs according to some user-defined
probability (usually high) and results in new chromosomes having
characteristics taken from both of the parent chromosomes. The most
common crossover operator is 뱋ne point crossover? This crossover
operator picks a random point within the chromosomes, then switches the
genes of the two chromosomes at this point to produce two new offspring.
If an offspring takes the best parts from each of its parents, the
result will likely be a better solution.
- Mutate the genes of the offspring chromosomes. This mutation occurs
according to some user-defined probability (usually low) and serves to
introduce some variability into the gene pool. Without mutation,
offspring chromosomes would be limited to only the genes available
within the initial population. For real genes, one of the most common
mutation operators simply adds a Gaussian distributed random value to
the gene being mutated.
- Repeat steps 3 through 5 until a new population of chromosomes has
been created. Two chromosomes are selected, crossed-over, and mutated to
produce two offspring until there are enough offspring to fill a
population the same size as the last population.
- Evaluate each of the chromosomes in the new population. Each of the
chromosomes in the current population is evaluated by the fitness
function, just as the initial population was evaluated in step 2.
- Repeat steps 3 through 7 until some termination condition has been
met. This termination condition can be based simply on the number of
generations or it can be based upon more complex criteria such as the
population convergence, fitness convergence, etc.
After the evolution of the initial population through many generations,
the chromosomes (or solutions) within the final population will generally
be much better as a whole than the chromosomes within the initial
population. Also, the best chromosome in the final population will
generally be near optimal if the genetic algorithm was run for enough
generations.
Return to
Topic List
Q.
How can genetic algorithms be used to
improve neural networks?
A. Genetic algorithms can be combined with neural
networks to enhance their performance by taking some of the guesswork out
of optimally choosing neural network parameters, inputs etc. In general,
genetic algorithms can be used in conjunction with neural networks in the
following four ways. They can be used to choose the best inputs to the
neural network, optimize the neural network parameters (such as the
learning rates, number of hidden layer processing elements, etc.), train
the actual network weights (rather than using backpropagation), or to
choose/modify the neural network architecture. In each of these cases, the
neural network error is the fitness used to rank the potential solutions.
Return to
Topic List
Q.
What are some of the types of neural
networks that I can build with NeuroSolutions?
A. NeuroSolutions includes two wizards to create
neural networks for you based on your data and specifications. One of
these wizards, the NeuralBuilder, centers the design specifications around
the specific neural network architecture you wish to have built. Some of
the most common neural network architectures include:
- Multilayer Perceptron (MLP)
- Generalized Feedforward
- Modular
- Jordan/Elman
- Principal Component Analysis (PCA)
- Radial Basis Function (RBF)
- General Regression Neural Network (GRNN)
- Probabilistic Neural Network (PNN)
- Self-Organizing Map (SOM)
- Time-Lag Recurrent Network (TLRN)
- Recurrent Network
- CANFIS Network (Fuzzy Logic)
- Support Vector Machine (SVM)
Return to
Topic List
Q.
What if I know nothing about the
various types of neural networks?
A. The second of the two network construction
wizards is called the NeuralExpert. The NeuralExpert centers the design
specifications around the type of problem you wish the neural network to
solve (Classification, Prediction, Function Approximation or Clustering).
Given this problem type and the size of your data set, the NeuralExpert
intelligently selects the neural network size and architecture that will
most likely produce a good solution. There is an optional beginner level
that hides some of the more advanced operations such as cross validation
and genetic optimization. In other words, you don't have to be a neural
network expert to develop successful models with NeuroSolutions!
Return to
Topic List
Q.
What if the neural network type I am
interested in is not listed?
A. NeuroSolutions was designed with flexibility
in mind. It is based on the concept that neural networks can be broken
down into a fundamental set of neural components. The network construction
wizards will connect these components for you based on your
specifications. However, once the network is built you can arbitrarily
change interconnections and/or add in new components. In other words, a
virtually infinite number of neural models are possible!
In addition, the Developers and Developers Lite levels
allow you to integrate your own algorithms into NeuroSolutions through
dynamic link libraries (DLLs). Every NeuroSolutions component implements a
function conforming to a simple protocol in C. To add a new component you
simply modify the template function for the base component and compile the
code into a DLL -- all directly from NeuroSolutions!
Return to
Topic List
Q. Does NeuroSolutions work with Excel
files?
A. The best way to develop a neural network model
from your Excel data is to use to use NeuroSolutions for
Excel. This Microsoft Excel add-in simplifies and enhances the process
of getting data into and out of a NeuroSolutions neural network. It
benefits both the novice and the advanced neural network developer by
allowing them to do everything from preprocessing the data, to designing
the neural network, to generating reports of the results -- all directly
from Excel!
If you are not interested in this add-in product you can still use your
Excel data within NeuroSolutions. All you need to do is save a copy of
your Excel spreadsheet as a text file (e.g., *.txt, *.csv).
Return to
Topic List
Q.
Once I develop a working neural
network in NeuroSolutions, how do I use it?
A. Once you've created your neural network and
trained and tested it on the historical data, you are now ready to put the
neural network to use. What you want to do is give the neural network
current input data as it becomes available, and have it generate an output
that you can utilize. There are several ways you can interface with your
NeuroSolutions neural network to put it into production:
- Within the Graphical User Interface of NeuroSolutions
The
TestingWizard automates the process of putting input data into a
NeuroSolutions neural network and getting data out. You simply select
the file that contains your new data and it will generate the network
output, either to a file or to the screen.
- Within your Excel Spreadsheet
NeuroSolutions
for Excel makes it very easy to put your neural network to use. You
simply add the rows of new data to your spreadsheet, tag them as
"Production" then select a menu item which will automatically to feed
the input data to the neural network and write the network outputs
directly to the output column(s) of the spreadsheet.
- Within your Custom Application
If your goal is to develop
an application that is based on neural network technology, then you will
need to be able to be able to interface the application with your
NeuroSolutions neural network. There are three ways you can use
NeuroSolutions to deploy your custom neural network solution into your
application:
- Dynamic Link Library (DLL) - The Custom Solution
Wizard is an optional add-in product that will take a neural
network designed within NeuroSolutions and encapsulate it into a
dynamic link library (DLL) that conforms to a simple protocol. This
DLL can then be embedded into your own C++, Visual Basic, Excel,
Access or Web (ASP) application. The key advantage to this approach is
that you don't need to be an advanced programmer to use it.
- C++ Code Generation - The Professional and Developers levels
of NeuroSolutions allow you to automatically generate C++ source code
for your neural network. This gives you the flexibility you may need
to customize the neural network code for your particular application.
Since the generated code is ANSI-compliant, you can deploy your neural
network solution to other platforms such as UNIX (requires the Source
Code License).
- OLE Automation - You as a programmer can have full control
of NeuroSolutions from an external application developed using an
Automation-supported platform such as Visual C++, Visual Basic,
Microsoft Access and Microsoft Excel. Because of its extensive API,
NeuroSolutions could be instructed to exchange data or perform any
series of operations programmatically that you could do manually from
the program's interface.
Return to
Topic List
Q. What Operating Systems does NeuroSolutions run
on?
A. The graphical user interface (GUI) of
NeuroSolutions requires Windows 95, 98, NT, 2000, Me or XP.
NeuroSolutions neural networks can run under other operating systems
such as UNIX, but without the GUI environment. The products required to
run a NeuroSolutions neural network on a platform other than Windows are
the Professional or Developers level of
NeuroSolutions and the Source Code
License. The process would be to design the neural network using a
Windows computer, then generate the C++ source code for the neural network
and compile that code on the target operating system.
|
