Towards machine learning

Question 1

Long-term memory and short term meory usage in machine learning

Answer 1

Machine learning algorithm typically accepts an input vector and produces output vector. To transform input vector to output vector, two additional vectors can be used. These additional vectors are referred to as long-term and short term memory.

Long term memory also referred to as weighted or coefficients. It is adjusted through training. short term memory is not used by all machine learning algorithms.

It might be helpful to think of a machine leraning algorithm as function. we will treat very simple equation as though it were a machine-learning algorithm in equation.

Question 2

Simple Linear equation ?

Answer 2

f(x)=5x

Here we think of X as a single value- a scalar rather than a vector. The value 5 is coefficient. Coefficients usually grouped into vector and represent the long term memory of the algorithm. When we train above equation we will adjust the coefficient untill we arrive at a value that produces the desired output.

Question 3

explain below input of simple linear equation

Input[1], desired Output: [7]

Input[2], desired Output:[14]

Input[3]: desired output: [21]

Answer 3

In the given example, the coefficient 5 would not provide the desired output. we could calculate the error between the desired outputs and actual outputs provided . Hene 7 is the final coefficient to desired output.

There are many different ways to find a suitable coefficient for machine learning algorithms. Coefficient adjustment methods is one of the primary areas of research for machine learning

Question 4

Why it is extremely rare to fit the coefficients ?

Answer 4

It is extremely rare to fit the coefficients (or long-term memory vectors) to result in an error that exactly zero. This is because data contains noise. Noise is any outcome that is not consistently reproducible given its input data and thus contributes to higher error rate.

Question 5

Goal of ML with noise in data

Answer 5

The goal of a ML algorithm is usually for the algorithm to perform well on new data, rather displaying zero error. If you do get a zero error then coefficients are over fit. Over fitting occurs when MLA memorized the training data.

Question 6

Coefficients of a polynomial?

Answer 6

Coefficients of polynomial is one of the methods for optimizing coefficients to lower the error. We can use polynomial as machine learning algorithm that we seek to optimize. A polynomial is mathematical expression with variables and constant coefficients and uses only the operations of additions,subtraction, multiplication and positive integer exponents.

Question 7

Greedy Random training?

Answer 7

1. There are many different ways to adjust long-term memory of a machine learning algorithm during training.
2. This algorithm is greedy in the sense that it accepts it only accepts new memory vector if the new vector is an improvement over the previous one.
3. Always maintains the best vector seen so far. Often this algorithm called a random walk.
4. The score is the key part to identify the improvement. It checks the score of the vector is compared to the score before randomization. If the score is decreased for new vector, then algorithm still considers previous one.
5. The main disadvantages of greedy training is no refinement.

Question 8

RBF network (Radial basis networks)

Answer 8

1. RBF network is a statistical model that can be used both for regression.
2. In RBF network, Long-term memory is a combination of coefficients and other parameters, there is no short-term memory.
3. It can be trained by both greedy and hill climbing algorithm based on Radial basis functions.

Question 9

Radial basis functions

Answer 9

1. An important concept in AI
2. It is symmetric about its center typically located somewhere along the x-axis. It reaches the maximum value at center.
3. The output of RBF always single scalar value regardless of the number of dimensions in the vector passed to RBF

Question 10

Gaussian function

Answer 10

1. In Gaussian RBF, accepts multi-dimensions. It is important to note that RBFs accept multi-dimension input and returns a scalar value, It does by calculation distance between input and center vector of RBF. The distance is called r.
2. Once “R” calculated we can calculate individual RBF functions by using “r”

Question 11

Various types of RBF functions?

Answer 11

1. Gaussian RBF function
2. Ricker wavelet function (Mexican Hat function)
3. MultiQuadric
4. Inverse quadratic
5. Inverse multi quadric
6. Polyharmonic spline
7. Thin plate spline

Question 12

Radial basis function network?

Answer 12

1. using RBF we can implement statistical model called RBF network.
2. we can train the specified model using above specified techniques
3. It can be used for both classification and regression.
4. It provides the weighted summation of one or more radial basis functions, each of which receives weighted input attributes to predict the output.

Question 13

Implementing RBF network?

Answer 13

In order to implement RBF network, we must provide three pieces of information.

Input count

RBF count

Output Count

Question 14

Long term memory in RBF network ?

Answer 14

Input coefficients

Out coefficients

RBF width scalars

RBF center vectors