Lecture 4

Question 1

What does GP stand for

Answer 1

Genetic Programming

Question 2

What does the genotype of a GP represent?

Answer 2

A program or math equation.

Question 3

How is the fitness of a GP solution determined?

Answer 3

By executing the program/solution and scoring the output. Usually the output is compared to a reference using MSE.

Question 4

What is the core principle of Symbolic Regression?

Answer 4

Given a data set of d features and n samples, where each sample of features results in y, find a function that can recreate this behaviour.

AKA find the hidden function given some input and output.

Question 5

What is the main reason to pick GP over ML?

Answer 5

GP will create interpretable models.

Question 6

In tree-based GP, what are the leave nodes called?

Answer 6

Terminal Nodes

Question 7

In tree-based GP, what are the internal nodes called

Answer 7

Atomic Funcions

Question 8

In tree-based GP, what is the primitive set? and where is it used?

Answer 8

Union of both the function set and the terminal set.

It is used in tree initialisation

Question 9

In tree-based GP, what is the terminal set?

Answer 9

A set of all terminal (leave) nodes. In GP, usually these are real values.

Question 10

Explain how a GP tree is initialised using the grow method.

Answer 10

• Start from root node
• If max depth has not been reached: randomly sample from primitive set.
• else: randomly sample from terminal set
• Stop when:
  
  • Max depth reached
  • Leaf node sampled

Question 11

Explain how a GP tree is initialised using the full method.

Answer 11

• Start from the root node.
• if max depth has not been reached: choose from function set
• else: randomly choose from terminal set
• Initialisation will stop itself as all leaves will be at max depth

Question 12

Explain how a GP tree is initialised using Ramped Half and Half.

Answer 12

Population is segmented into bins where every bin has its own max depth. The max depth ramps up per bin, such that it covers all depths between max and min depth.

Furthermore, each bin is initialised with ** both grow and full** method. It randomly selects a method for each individual.

Duplicates within a bin are discarded

Question 13

In tree-based GP, what is non injective?

Answer 13

Many different trees with the same output can be generated.

Question 14

Does a GP tree use crossover of mutation to cause variation?

Answer 14

Both.

Question 15

How does a GP mutate?

Answer 15

It generates a new subtree, with wich a branch is replaced.

Question 16

Why does GP require linear scaling?

Answer 16

The hardest part is getting the shape of your Symbolic Regression model right. If the MSE is very large because the model is displaced or of the wrong magnitude, this would take the GP very long to fix. However it can easily be solved mathematically, and hence be crucial for computation time.

Question 17

What type of selection is often used in tree-base GP?

Answer 17

Tournament selction, with large tournament size.

Question 18

When does GP meet sufficiency?

Answer 18

When the primitive set is sufficient; which is when it allows to encode the optimal program.

Question 19

When does GP meet consistency?

Answer 19

When the types of the primitive set + variation operators are type consistent. (f.i. you cannot apply boolean operators on real values.)

Question 20

What strategies are there for consistency?

Answer 20

1. Flexibilty in function types
2. Put constraint on variation operators
3. Repair invalid subtrees.

Question 21

When does GP meet evaluation safety?

Answer 21

If all elements of the primitive set are evaluation-safe: when the evaluation of an atomic function does not comprise the evolution.

F.i. a division is comprimised when the denum is zero.

Question 22

How can we ensure evaluation safety?

Answer 22

Implement protected operators, often by introducing epsilon.

Question 23

What is an ERC and where is it used.

Answer 23

Ephemeral Random Constant is used to denote real values at the initialisation of trees. They are randomly selected between the bounds.

Question 24

In GP, what is bloating?

Answer 24

When the tree grows (becomes more complex) without much increase in accuracy.

Question 25

What are the shortcommings of GP?

Answer 25

1. Unknown if optimal has been reached
2. Single output
3. bloating

Question 26

In GP, how can we combat the shortcomming of accuracy suffering due to size limitation?

Answer 26

1. Apply scaling
2. adding more operators
3. Apply pattern matching to simplify tree.
4. Coefficient optimalisation: Where you replace subtrees with constants.

Question 27

What is a solution for GP bloating and when is it usefull?

Answer 27

Exhaustive Symbolic Regression is when you take a more bruteforce approach. It can be usefull when you dont care about time, f.i. space exploration.

Question 28

What is a solution for multiple output in GP trees, and how does it work?

Answer 28

Cartesian Genetic Programming. function nodes are in a grid and their output value is labeled, such that other internal nodes can connect to it. Function nodes can only connect with every prior node, even leaves.

Question 29

Expressions resulting from GP are always interpretable.

Answer 29

Nah-ah