Evaluation Flashcards
Evaluation
Evaluation is the process of understanding the reliability of any AI model, based on outputs by feeding test dataset into the model and comparing with actual answers.
underfitting
- The model’s output does not match the true function at all.
-accuracy is lower.
perfect fit.
he model’s performance matches well with the true function which states that the model has optimum accuracy
overfitting
model performance is trying to cover all the data samples even if they are out of alignment to the true function
a lower accuracy.
To understand the efficiency of any model
two conditions
Prediction and Reality
prediction
The prediction is the output which is given by the machine and the reality is the real scenario on which the prediction has been made.
confusion matrix
The result of comparison between the prediction and reality can be recorded in what we call the confusion matrix. The confusion matrix allows us to understand the prediction results
Which Metric is Important?
OR
Which value for the F1 score is the perfect F1 score? Explain with context.
F1-Measure provides a single score that balances both the concerns of precision and recall in one number. A good F1 score means that you have low false positives and low false negatives, so you’re correctly identifying real threats, and you are not disturbed by false alarms.
An F1 score is considered perfect when it’s 1, while the model is a total failure when it’s 0. F1 Score is a better metric to evaluate our model on real-life classification problems and when imbalanced class distribution exists.
In conclusion, we can say that a model has good performance if the F1 Score for that model is high.
What is the purpose of evaluation?
Evaluation is a process that . It involves collecting and analysing information about a program’s activities, characteristics, and outcomes. Its purpose is to make judgments about a program, to improve its effectiveness, and/or to inform programming decisions.
What is the importance of evaluation?
- Evaluation is important to ensure that the model is operating correctly and optimally.
- Evaluation is an initiative to understand how well it achieves its goals.
- Evaluations help to determine what works well and what could be improved in a program
. Explain the evaluation metrics for
a)mail spamming
In mail spamming, if the machine predicts that any email is spam email, then the person will ignore that email. In this context sometime the person miss out the vital information. False positive condition would have high cost as predicting the email as spam while the mail is not a spam email.
. Explain the evaluation metrics for
b) gold mining
A model predicts that there exists gem at a point and the individual keep on excavating there but it turns out that it is a false apprehension. False positive condition is very costly as predicting there is a treasure but in reality there is no treasure.
