Evaluation-1

Question 1

what is reliability

Answer 1

Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding the test data set into the model and comparing with actual answers.

Question 2

why us model evaluation importiannt

Answer 2

Model Evaluation is an integral part of the model development process. It helps to find the best model that represents our data and how well the chosen model will work in the future.

Question 3

define undeffittig

Answer 3

In the first diagram,The model’s output does not match the true function at all. Hence the model is said to be underfitting and its accuracy is lower.

Question 4

define overfitting

Answer 4

When we use the same data we used to build the model for evaluaion,the model will simply remember the whole training set, and will therefore always gives correct prediction. This is known as overfitting.
In the third case, model performance is trying to cover all the data samples even if they are out of alignment to the true function. This model is said to be overfitting and this too has a lower accuracy.

Question 5

what is a perfect fit

Answer 5

the model’s performance matches well with the true function which states that the model has optimum accuracy and the model is called a perfect fit.

Question 6

what is
i) prediction
ii) reality

Answer 6

i) prediction is the output that is given by the machine
ii) reality is the true scenario wich takes place for which the prediction was made.

Question 7

what is a confusion matrix?

Answer 7

it table setup which Helps to understand the prediction results. It shows the comparison between the prediction and reality .This is sometimes referred to as Contingency table or error matrix or matching matrix.

A 2x2 matrix denoting the right and wrong predictions might help us analyse the rate of success. This matrix is termed the Confusion Matrix.

Question 8

what is true positive

Answer 8

The predicted value matches the actual value

The actual value was positive and the model predicted a positive value

Question 9

what is true negative

Answer 9

The predicted value matches the actual valueThe actual value was negative and the model predicted a negative value

Question 10

what is false positive

Answer 10

The predicted value was falsely predictedThe actual value was negative but the model predicted a positive valueAlso known as the Type 1 error

Question 11

what is false negative

Answer 11

The predicted value was falsely predictedThe actual value was positive but the model predicted a negative valueAlso known as the Type 2 error

Question 12

what are the 4 evaluation meaures

Answer 12

-accuracy
-precision
-recall
-F1 score

Question 13

define accurucay

Answer 13

Accuracy is defined as the percentage of correct predictions out of all the observations. A prediction can be said to be correct if it matches the reality. Here, we have two conditions in which the Prediction matches with the Reality: True Positive and True Negative.
accuracy= correct predicitoon/total case x 100
=tp+tn/tp+tn+fp+fn x 100

Question 14

why is accuracy not a good measure

Answer 14

Assume that the model always predicts that there is no fire. But in reality, there is a 2% chance of forest fire breaking out. In this case, for 98 cases, the model will be right but for those 2 cases in which there was a forest fire, then too the model predicted no fire.
Here,
True Positives = 0
True Negatives = 98
Total cases = 100
Therefore, accuracy becomes: (98 + 0) / 100 = 98%
This is a fairly high accuracy for an AI model. But this parameter is useless for us as the actual cases where the fire broke out are not taken into account. Hence, there is a need to look at another parameter which takes account of such cases as well.

Question 15

define precision

Answer 15

Precision is defined as the percentage of true positive cases versus all the cases where the prediction is true. That is, it takes into account the True Positives and False Positives.
precision= tru positive/ all predicted positivies x 100
= tp/tp+fp x 100

Question 16

why is precision important

Answer 16

You might recall the story of the boy who falsely cries out that there are wolves every time and so when they actually arrive, no one comes to his rescue. Similarly, here if the Precision is low (which means there are more False alarms than the actual ones) then the firefighters would get complacent and might not go and check every time considering it could be a false alarm.
This makes precision an important criteria.

If precision is high, this means the true positive cases are more, giving lesser false alarms.

Question 17

what is recall

Answer 17

It can be defined as the fraction
of positive cases that are correctly identified. It majorly takesinto account the true reality cases where
in Reality there was a fire but the machine either detected it correctly or it didn’t. It considers tp and fn

recall= tp/tp+fn

Question 18

when do we use precision and recal

Answer 18

when false negative costs more- recal
when false positive costs more-precision

Question 19

what is F1 score?

Answer 19

F1 score can be defined as the measure of balance between precision and recall.
F1= 2 x precision x recall/ precision+recall

Question 20

what is an ideal f1 score

Answer 20

An ideal situation would be when we have a value of 1 (that is 100%) for both Precision and Recall. In
that case, the F1 score would also be an ideal 1 (100%). It is known as the perfect value for F1 Score.
As the values of both Precision and Recall ranges from 0 to 1, the F1 score also ranges from 0 to 1.