Lecture 16 - Genetic Algorithms Flashcards
What are Genetic Algorithms?
- Inspired by Darwinian evolution and modern genetics.
- Core operators:
○ Mutation
○ Crossover (recombination/mating)
○ Selection
What are the Key Differences from Evolution Strategies (ES)?
- GA focuses on individual vectors as chromosomes/genomes, traditionally in binary encoding.
- Crossover is essential, not just mutation.
- Various methods for parent selection, often probabilistic.
What are the Similarities to (µ,λ) ES?
They are similar to (µ,λ) ES, but
- focus on individual vector as chromosome/genome
○ traditionally binary encoding not real/floating point (but of course… )
- crossover is a fundamental operation (not only mutation)
- many approaches to parent selection (sometimes probabilistic)
How do Genetic Algorithms Work?
- Start with a bunch of random solutions.
- Keep the best one seen so far.
- Select good parents.
- Mix and mutate them to create new ones.
- Repeat — improving each generation.
Genetic Algorithm Code
REFER TO SLIDES
What are the Key Differences from (μ, λ) Evolutionary Strategy?
- Uses crossover as a core operator (vs. just mutation)
- Parents are selected probabilistically, not just top μ
- Creates a new population from parent pairs rather than selecting best individuals directly
- Follows a “generate and replace” model using selection → crossover → mutation
What is Initiasation?
- “Before we can evolve anything, we need a starting population — and that starts with initialising individuals.
- Explains how to generate random binary vectors, which are the most common encoding in basic GAs.
- But it also reminds us that binary is not the only option — GAs can use other representations (e.g., trees, real-valued vectors, or even chemical compounds!).
Initialisation Code - why use binary?
REFER TO SLIDES
- Using binary vectors (remember there are other ways to do this)
- Why use binary?
○ Easy to mutate (flip bits)
○ Simple crossover logic
○ Common in theoretical work
Mutation Code
REFER TO SLIDES
What is One-Point Crossover?
One-point crossover is a method of combining two parent chromosomes to produce offspring by swapping parts of their genetic information after a randomly selected crossover point.
- Imagine two binary strings (chromosomes). We pick a random position in the middle, and we swap everything after that point between the two parents. This creates two new children that are combinations of both parents.
One-Point Crossover Code
REFER TO SLIDES
What are the Disadvantages of One-Point Crossover?
REFER TO SLIDES - NEED FORMULAS TO EXPLAIN
What is Bias Recombination?
“The algorithm is biased against combining genes that are far apart. This can slow down convergence or stop the GA from finding optimal combinations.”
That’s why this slide notes:
- “We’re biasing against solutions where distant genes work well together.”
What is Two-Point Crossover?
- Two-point crossover is a genetic operator that takes two parent chromosomes and swaps the segment between two randomly chosen points. It’s more balanced than one-point crossover and can better preserve building blocks of genes — even if they’re far apart
Two-Point Crossover Code
REFER TO SLIDES
Two-Point Crossover - Further Details Through Visualisation
REFER TO SLIDES
What is Uniform Crossover?
- Uniform crossover decides whether to swap genes at each position individually, instead of cutting the chromosome at fixed points. It gives every gene position an equal, independent chance to be swapped.”
○ This removes the positional bias seen in one-point and two-point crossover, and provides maximum mixing potential.
Uniform Crossover Code
REFER TO SLIDES
Why is Crossover is Not Mutation?
- Crossover recombines existing values — it does not invent new values like mutation can.
○ Core issue: If the initial population lacks diversity, crossover alone can’t fix it. You may never explore some parts of the solution space. - Expands the issue to geometry:
○ Binary chromosomes of length l sit on the corners of an l-dimensional hypercube. Crossover only moves you around that existing space — it cannot escape it.
○ Dangers:
* If all individuals have the same bit in one position, crossover can’t restore variation there.
* That dimension is collapsed, and permanently lost.
* Eventually, everyone converges to a single solution — even if it’s not optimal.
○ This leads to:
* Premature convergence
* Loss of diversity
* Reliance on mutation to restore variability
So why do we use Crossover?
- Just because its in “biology” doesn’t mean it makes for the best computer algorithm, so more justification is needed
- Argument for Crossover:
○ It spreads good building blocks (short gene patterns that work well).
○ This idea is supported by:
* Empirical evidence (some success in practice)
* Schema Theory (a theoretical model explaining how gene patterns evolve)
○ But…
* Schema theory is complex and hard to generalize
* It’s not fully satisfying in explaining why crossover works reliably
- Argument for Crossover:
Why does Crossover rely on the assumption of Epistasis?
Linkage is key:
* One-point crossover: depends on position
* Two-point crossover: depends on distance
* Uniform crossover: depends on probability p
What is Selection?
- “Selection is the process of choosing which individuals in the population will become parents and produce offspring for the next generation.”
○ Selection influences how fit individuals are rewarded and how diversity is maintained. - Recall truncation selection in ES’s
REFER TO SLIDES ABOUT THIS
- SelectWithReplacement is open to a wide range of options
What is Fitness-Proportionate Selection?
- Original and common approach
○ Also known as roulette selection
* “Roulette selection is a method where individuals are chosen to reproduce with a probability proportional to their fitness.
* It’s like spinning a weighted roulette wheel — the more fit an individual is, the bigger their slice of the wheel, and the more likely they are to be selected.”
==== - Select individuals in proportion to their fitness
○ “If an individual has twice the fitness of another, it should be twice as likely to be chosen.”
Fitness-Proportionate Selection Code
REFER TO SLIDES
What is the time for Fitness-Proportionate Selection?
○ “This algorithm runs in O(n) time for each selection — but with binary search on the CDF, it can be improved to O(log n).”
What is Stochastic Universal Sampling?
- “Stochastic Universal Sampling is a low-variance selection method that still chooses individuals based on fitness, but does so in a more evenly distributed way.
○ It helps reduce randomness and makes the selection process fairer and more consistent.”
How are pointers spaced in Stochastic Universal Sampling?
How Pointers Are Spaced
* Total range = s (sum of all fitness)
* Number of individuals to choose = n
* Each pointer is spaced s/n apart
* First pointer starts randomly in the range [0, s/n]
“This ensures coverage of the whole fitness space, not just the parts with the highest fitness.”
What are the Advantages of Stochastic Universal Sampling
- Low variance resampling
- Advantage: Reduces randomness in the selection process compared to methods like roulette wheel selection.
- Leads to more stable and consistent selection of individuals over generations.
- Guarantees selection of individuals with fi≥snf_i \geq \frac{s}{n}fi≥ns
- Advantage: Ensures that individuals with high fitness are always selected, which promotes survival of the fittest.
- Leans toward exploitation
- Advantage or trade-off: Since it focuses more on selecting fitter individuals, it preserves good solutions.
- Potential downside only if you’re in a landscape where exploration is critical — in such cases, diversity may be reduced, leading to premature convergence.
What is Tournament Selection?
- “Tournament selection is a non-parametric selection method — it doesn’t care about the actual fitness values, only who’s better.
- We run a mini-competition (‘tournament’) among a few individuals and select the best one from that group.”
○ It’s easy to implement.
○ Works well even if fitness values are noisy, close together, or poorly scaled.
Tournament Selection Code
REFER TO SLIDES
Why is Tournament Selection used so commonly?
“Not sensitive to size of fitness values”
“Unlike roulette or SUS, tournament selection only cares about who’s better, not how much better.
That makes it immune to weird scaling or small differences in fitness.”
====
“Simple (and easily parallelised)”
“Tournament selection is very easy to implement and fast.
You can also run many tournaments at the same time — which makes it ideal for parallel processing in large-scale applications.”
====
“Tunable by setting tournament size t”
The beauty of tournament selection is that you can control how aggressive it is by changing t, the tournament size.
🔸 When t = 1
“You just pick one random individual — no competition.
This is like pure random selection, with no selection pressure at all.”
What is Elitism?
Elitism is the idea that if you already have a really good solution, you should keep it!
○ So, you automatically carry over the best individual(s) to the next generation, without any modification or competition.”
- These individuals are called the ‘elite group’
* They’re protected from mutation, crossover, or being replaced.
* This guarantees that the best solution so far is never lost.
Why use Elitism?
- Why Use Elitism?
- “It’s a form of exploitation — making sure we don’t throw away a good thing.
- It’s like saying: ‘We already found gold — don’t forget where we buried it!’”
- Similar in spirit to (μ + λ) Evolution Strategies, where parents can survive into the next generation.
- Helps improve convergence speed because the population keeps getting better (or at least not worse).
What is the catch with Elitism?
“Too much elitism can be dangerous!”
Problem: Premature Convergence
“If we only focus on elites, the population may lose diversity — and all individuals may become nearly identical.
That can lead to getting stuck in a local optimum.”
How can you keep Elitism in check?
How to Keep Elitism in Check
The slide suggests three strategies:
1. Increase mutation
“Add more randomness to help escape local optima and reintroduce variety.”
2. Breed in diversity from outside the elite group
“Allow less-fit individuals to contribute genes — they might introduce useful traits.”
3. Reduce how many elites are stored
“Only preserve the top 1 or 2 — not the entire top half of the population.”
Elitism Code
REFER TOP SLIDES
