M2.2 Flashcards
Three possible practical implementations of ANNs are:
1. A software simulation program running on a digital computer.
2. A hardware emulator connected to a host computer - the so-called _____.
3. A true electronic neural network
neurocomputer
This is currently the cheapest and fastest implementation method for ANNs
Software Simulations
Software Simulations simulates parallel processing on a conventional ___ digital computer
sequential
Software Simulations replicates ___behaviour of the network by updating the activation level and output of each node for successive time steps
temporal
Typical additional features of ANN simulators
1. Configuring the net according to a chosen architecture and node operational characteristic
2. Implementation of ___ phase using a chosen training algorithm
training
ANN simulators are written in ___ languages such as C, C++ and Java
hi-level
Main attraction of ANN simulators is the relatively low cost and wide availability of ready made commercial ___
packages
ANN simulators are also compact, flexible and highly ___
portable
Training of ANNs using software simulators can be slow for larger networks (greater than a few ___)
hundred
Commercially Available Neural Net Packages have prewritten ___ with convenient user interfaces
shells
Commercially Available Neural Net Package cost a few ___ of dollars
hundred to tens of thousands
Commercially Available Neural Net Package allow users to specify the ANN design and training ____
parameters
Commercially Available Neural Net Package usually provide ___ to enable monitoring of the net’s training and operation
graphic interfaces
Commercially Available Neural Net Package are likely to provide interfacing with other software systems such as ___ and databases
spreadsheets
___ are dedicated special-purpose digital computer (aka accelerator boards
neurocomputer
Neurocomputers are dedicated special-purpose digital computer (aka ___)
accelerator boards
Neurocomputers acts as a ___to a host computer and is controlled by a program running on the host
coprocessor
Neurocomputers can be tens to ___ of times faster than simulators
thousands
Neurocomputers systems are available with over 10 million ___ for networks with several million neurons
IPS
In neurocomputers, IPS means?
IPS (Interconnect updates Per Second)
___ in hardware are closer to biological neural networks
True Networks
True Networks in Hardware are consist of synthetic neurons actually fabricated on ___ chips
silicon
Commercially available ___ ANNs are limited to a few thousand neurons per chip
hardwired
The problem with true networks in hardware wherein it is hard to make all the neurons communicate properly across chips
Interconnection and interference issues
The problem with true networks in hardware wherein some chips can’t change their “memory strength” after being made
fixed-valued weights
The problem with true networks in hardware wherein scientists are still working on making the connections changeable or a modifiable ___
synapses
This aims to add structure and organization to ANN applications development for reducing cost, increasing accuracy, consistency, user confidence and friendliness
Neural Network Development Methodology
Neural Network Development Methodology are split into (4) phases:
The Concept Phase
The Design Phase
The Implementation phase
The Maintenance Phase
In Neural Network Development Methodolog, this phase involves
o Validating the proposed application
o Selecting an appropriate neural paradigm.
The Concept Phase
One application area unsuitable for ANNs is ___ management eg, inventory, accounts, sales data analysis
resource
Selecting an ANN Paradigm is based on comparison of application requirements to capabilities of different paradigms as well as the ___ method that can be employed
training
This phase of Neural Network Development Methodology specifies initial values and conditions at the node, network and training levels
The Design Phase
In Node-Level Decision of the the Design Phase, this type of input is like a light switch: ON or OFF (0,1)
Binary
In Node-Level Decision of the the Design Phase, is like YES/NO with more balance (-1, +1)
Bipolar
In Node-Level Decision of the the Design Phase, this type of input has three options: low, neutral, high (-1, 0, +1)
Trivalent
In Node-Level Decision of the the Design Phase, these are the other two types of input (2)
Discrete, Continuous
In Node-Level Decision of the the Design Phase, what is this type of Transfer Function
If input is big enough, turn ON; else, stay OFF
Step/Threshold
In Node-Level Decision of the the Design Phase, what is this type of Transfer Function
Smooth curve output between 0 and 1
Sigmoid
In Node-Level Decision of the the Design Phase, what is this type of Transfer Function
Smooth curve from -1 to +1
Tang (hyperbolic tangent)
In Node-Level Decision of the the Design Phase, what is this type of Transfer Function
Precomputed answers to speed things up
Lookup Table
In Network-Level Decision of the the Design Phase, __ is described as how many inputs you give (like pixels of an image)
Input layer size
In Network-Level Decision of the the Design Phase, __ is described as how many categories or clusters you expect
Output layer size
In the Design Phase, this type of connectivity is when neurons only connect between layers (like in Multilayer Perceptron or MLP)
Interlayer Connectivity
In the Design Phase, this type of connectivity is when neurons also connect within the same layer (like in Hopfield networks)
Intralayer Connectivity
In the Design Phase, __ feedback is once the signal moves forward, it doesn’t go back (like in MLP)
no feedback
In the Design Phase, __ feedback is when outputs can be sent back into the system to help adjust behavior (like in Hopfield net)
with feedback
This parameter in the Design Phase is defined as how fast the network updates its weights
learning rate
This parameter in the Design Phase is defined as when should training stop? (e.g., after 1000 rounds, or when error is small enough)
stopping rule
This parameter in the Design Phase is defined as adding small random numbers cto help the network learn faster (like shaking things up to avoid getting stuck)
adding noise
What is the last step in the implementation step of Neural Network Development Methodology
o Gathering the training set
o Selecting the developing environment
o Implementing the neural network
o ___
Testing and debugging the network
In gathering training data in the implementation step, Increasing data amount increases ___ time but may help earlier convergence
training
In gathering training data in the implementation step, quality more important than quantity. True or false
True
In the implementation step, when preparing for data, it usually includesnormalization and possible ___
binarization
In the implementation step, selecting the __ environment is also included. Such as picking the right hardware and software aspects
Development Environment
In the implementation phase, the most popular platforms to consider as the development environment are workstations and high-end PC’s (with ___ board option)
accelerator
In the implementation phase, when choosing the software, this type of software requires more expertise on part of the user but provides maximum flexibility
Custom-coded simulators
In the implementation phase, when choosing the software, this type of software are usually easy to use because of a more sophisticated interface
Commercial developement packages
This phase of Neural Network Development Methodology is consists of
o placing the neural network in an operational environment with possible integration
o periodic performance evaluation, and maintenance.
The Maintenance Phase
In the Maintenance Phase, when putting the Network to Work, this type of setup Works by itself, like a calculator
Stand-alone
In the Maintenance Phase, when putting the Network to Work, this type of setup Works with other systems across a network
Distributed
In the Maintenance Phase, when putting the Network to Work, this type of setup Prepares data for another system
Preprocessor
In the Maintenance Phase, when putting the Network to Work, this type of setup Processes the output from another system
Postprocessor
In the Maintenance Phase, when putting the Network to Work, this type of setup Built directly into another system (like a smart fridge or robot)
Embedded
In the Maintenance Phase, when putting the Network to Work, this type of connection to other systems is when ANN works alongside other systems but is not deeply connected. Examples: preprocessor, postprocessor, separate module
Loose-coupling
In the Maintenance Phase, when putting the Network to Work, this type of connection to other systems is when ANN is fully integrated into another system — it feels like part of the main program or machine
Tight-coupling
In the Maintenance Phase, the two main wais to AN maintenance is (2)
Data Maintenance
Software Maintenance
