Lecture 17 - Emergent Behaviour and Hybrid Algorithms Flashcards
What is Emergent Behaviour in Optimisation?
- Emergent behaviour refers to complex, organised behaviour that arises from the interactions of simple individual agents, without being explicitly programmed or centrally controlled.
- In the context of optimisation algorithms:
○ Each agent (e.g., particle, individual solution) follows simple rules.
○ Over time, group-level patterns emerge — such as convergence toward optimal solutions.
○ This is not coded directly, but is a natural consequence of agent interactions.
Real Wolrd Example of Emergent Behaviour in Optimisation
- Imagine a flock of birds searching for food:
○ Each bird decides where to go based on:
§ Its own experience (e.g., it found fish in one location),
§ The movements of nearby birds (others found better spots),
○ Eventually, the entire flock converges on the best fishing spot, even though no bird is in charge. - This is emergent behaviour — a smart global pattern from simple local rules.
Why is Emergent Behaviour in Optimisation Used?
- It avoids getting stuck in poor local optima (thanks to diversity and randomness).
- It adapts dynamically — the swarm learns from new discoveries.
- It scales well in complex, high-dimensional spaces where traditional methods (e.g., gradient descent) might fail.
What is Particle Swarm Optimisation?
- PSO is a population-based optimisation algorithm inspired by animal group behaviour, especially the swarming or flocking observed in:
○ Birds flying together
○ Fish schooling
○ In this example: a flock of seagulls fishing - The goal of PSO is to find the best solution (i.e., the “biggest fish”) by mimicking this collective search behaviour.
What is the core understanding of Particle Swarm Optimisation?
Swarming/Flocking Behaviour
- PSO is inspired by how animals like birds or fish collectively move in nature.
- Each individual makes independent decisions about where to go next, asking:
○ “Should I go to where I found fish before?” → (personal best)
○ “Should I go to where the biggest or most fish have been found?” → (global best)
○ “Should I explore a nearby promising spot?” → (local search)
○ Or use a combination of these? → Most PSO algorithms use a weighted combination of all.
- These decisions mimic real animal instincts and form the basis of how particles in PSO navigate the solution space.
What is Emergent Behaviour in Particle Swarm Optimisation?
- As each particle follows its own rules while adapting based on others, a global pattern emerges.
- Over time, the swarm converges on the best solution — the area with:
○ The highest concentration of good solutions.
○ Or the largest (most optimal) solution value. - This convergence is not hardcoded — it’s an emergent behaviour that results from local interactions.
- Emergent Behaviour = Complex global solution from simple local rules.
What is the Exploration vs Exploitation of Particle Swarm Optimisation?
Collective - Exploits known good areas (exploitation) -> Following global best
Individual - Explores new areas for better solutions (exploration) -> Momentum + personal best
- A good PSO implementation maintains this tension so the swarm doesn’t converge too early (getting stuck in a local optimum), or explore endlessly (never converging).
What are the building blocks of Particle Swarm Optimisation?
REFER TO SLIDES
How does Particle Swarm Optimisation work?
REFER TO SLIDES
Particle Swarm Optimisation Code
REFER TO SLIDES
What is PSO Personal vs Social in Particle Swarm Optimisation?
- It’s explaining two perspectives (or interpretations) of how a particle moves in Particle Swarm Optimization (PSO):
i. “Personal” vs “Social” – how a particle learns from itself vs others
What is PSO Cognitive vs Social in Particle Swarm Optimisation?
REFER TO SLIDES - HAS FORMULA
What is Differential Evolution (DE)?
- Works well in real-valued, multi-dimensional search spaces.
- Offspring are generated by vector recombination of three other population members.
- Mutation size depends on population diversity (adaptive).
- Children compete directly with parents—survival of the fittest.
- Promotes diversity-aware exploration.
What are Hybrid Optimisation Algorithms?
Hybrid Optimisation Algorithms — a powerful concept where we combine different types of optimisation methods to get the best of both worlds: fast local search and broad global exploration.
- More specifically:
“A hybrid algorithm is any algorithm that combines two or more different optimisation techniques to exploit their strengths and cover each other’s weaknesses.”
What is the problem with different optimisation algorithms?457
- “Different optimisation algorithms have different strengths — and weaknesses.”
○ Single-state algorithms (like hill climbing) are:
§ Fast and efficient at finding local optima
§ Great at exploitation (refining known good solutions)
§ But they struggle to escape local traps
○ Population-based algorithms (like Genetic Algorithms, Evolutionary Strategies, PSO):
§ Good at exploring the search space globally
§ Better at avoiding local optima
§ But often slower to converge precisely
Whats the general idea of Hybrid Optimisation Algorithms?
- “What if we combine the two?”
- We can use:
○ Population-based methods to explore broadly (e.g., GAs, PSO)
○ Local search methods like hill climbing to refine individuals once they’re close to something good
This creates a hybrid algorithm.
- We can use:
Hybrid Optimisation Algorithm Code
REFER TO SLIDES
What are Memetic Algorithms?
Genes vs Memes (Biology vs Culture)
Genes = biological inheritance (used in Genetic Algorithms)
Memes = cultural inheritance (ideas, behaviours passed between individuals)
“Memetic algorithms are inspired by the idea of memes — units of knowledge or behaviour that spread within a population.
Just like memes in real life, they can influence how individuals learn and adapt.”
What make Memetic Algorithms Different?
What Makes Memetic Algorithms Different?
“In regular evolutionary algorithms, individuals evolve only through genetic operations (mutation, crossover, selection).
But in memetic algorithms, individuals also undergo local learning or refinement, like hill climbing.”
So it’s like:
* “Evolution gives you your genes”
* “Memes help you learn, improve, or adapt after you’re born”
This allows for individual learning within a generation.
=====
“Memes” in Algorithms
* Memes spread across individuals (like knowledge in a team)
* This represents social learning, not just genetic inheritance
* It enables within-generation improvement (e.g., using hill climbing or knowledge transfer)
