Ensemble Methods and ANNs Flashcards
Why do ensemble methods work better?
Ensemble Methods are a way of combining the strength and accuracy of multiple classifiers in one model. They are ttained simultaneously on (a) given data set(s) and for each data instance the label is predicted as given by the majority of the individual classifiers. However these classifiers are not 100% independent, when using the same data set. This calls for a differentiation of the used data sets.
What is the difference between ‘by consensus’ and ‘by learning’?
Combining methods by consensus is done through ‘bagging’ and ‘random forests’. These are combination methods that use bootstrapping to come to multiple model outputs and then choose the corresponding label value for an instance bases on the majority classifier.
Combining methods by learning is done through boosting and stacking techniques. They use both bootstrapping and weight of the training instances to ‘learn’ which ones are often classified wrongly/correctly. Stacking is an advanced technique, as it uses an meta learner with base learner results as its inputs.
How does random forests work?
Random forests uses ensemble methods with T trees with each m of the total M variables. For each node T, the m features are used to find the best split and majority voting is used in every and among the trees to determine the predicted value for a new instance.
What are activation functions and why are there so many?
Activation functions are functions used in neural networks to determine an output value from the sum of all the inputs (weighted by weight factors). A particular sum corresponds to a more comprehensive value in e.g. a sigmoid (between 0 ane 1, like logistic regression) or a threshold function (either -1 or 1).
