Rule-Based Classifiers

Question 1

Rule-based classifier

Answer 1

A machine learning algorithm that makes predictions based on a set of explicitly defined rules, which are based on the relationships between the features and the target classes in the training data.

Question 2

Coverage of a rule

Answer 2

Fraction of records that satisfy the antecedent of a rule

Coverage (r) = |A| / |D|
|A| - The number of instances covered by the rules.
|D| - The total number of instances in the dataset.

Question 3

Accuracy of a rule

Answer 3

Fraction of records that satisfy the antecedent that also satisfy the consequent of a rule.

Accuracy (r) = |A n y| / |A|

Question 4

3 Type of Rule

Answer 4

Mutually exclusive rules - Every record is covered by at most one rule
Exhaustive rules - Each record is covered by at least one rule
Default rule - Not trigger any rules

Question 5

2 Type of Rule-Set

Answer 5

Ordered Rule Set - Rules ranked in decreasing order according to their priority
Rule-based Ordering Scheme - Ranked based on their quality.
Class-based Ordering Scheme - Shorter length, higher priority

Unordered Rule Set
Hard Voting (Majority Voting) - The highest number of votes.
Soft Voting (Weighted Voting) - Vote weighted by the rule’s accuracy..

Question 6

2 Rule Extraction Methods

Answer 6

Direct method - Extract rules directly from data
Indirect method - Extract rules from other classification models; E.g. Decision Tree

Question 7

4 Step of Direct Method - Sequential Covering

Answer 7

Start from an empty rule
Grow a rule using the Learn-One-Rule function
Remove training records covered by the rule
Repeat Step (2) and (3) until stopping criterion is met

Question 8

5 Aspects of Sequential Covering

Answer 8

Rule Growing
Instance Elimination
Rule Evaluation
Stopping Criterion
Rule Pruning

Question 9

3 Metric of Rule Evaluation

Answer 9

Accuracy = nc / n
Laplace = (nc + 1) / (n + k)
M-estimate = (nc + kp) / (n + k)