Service discovery Flashcards
Service discovery
Client-side discovery Server-side discovery Service registry Self registration 3rd party registration
Client-side discovery: context
Services typically need to call one another. In a monolithic application, services invoke one another through language-level method or procedure calls. In a traditional distributed system deployment, services run at fixed, well known locations (hosts and ports) and so can easily call one another using HTTP/REST or some RPC mechanism. However, a modern microservice-based application typically runs in a virtualized or containerized environments where the number of instances of a service and their locations changes dynamically.
Client-side discovery: problem
How does the client of a service - the API gateway or another service - discover the location of a service instance?
Client-side discovery: forces
Each instance of a service exposes a remote API such as HTTP/REST, or Thrift etc. at a particular location (host and port)
The number of services instances and their locations changes dynamically.
Virtual machines and containers are usually assigned dynamic IP addresses.
The number of services instances might vary dynamically. For example, an EC2 Autoscaling Group adjusts the number of instances based on load.
Client-side discovery: solution
When making a request to a service, the client obtains the location of a service instance by querying a Service Registry, which knows the locations of all service instances.
Client-side discovery: result benefits
Fewer moving parts and network hops compared to Server-side Discovery
Client-side discovery: result drawbacks
This pattern couples the client to the Service Registry
You need to implement client-side service discovery logic for each programming language/framework used by your application, e.g Java/Scala, JavaScript/NodeJS. For example, Netflix Prana provides an HTTP proxy-based approach to service discovery for non-JVM clients.
Client-side discovery: related
- Service Registry - an essential part of service discovery
- Microservice chassis - Client-side service discovery is the responsibility the microservice chassis framework
- Server Side Discovery is an alternative solution to this problem.
Server-side service discovery: context
Services typically need to call one another. In a monolithic application, services invoke one another through language-level method or procedure calls. In a traditional distributed system deployment, services run at fixed, well known locations (hosts and ports) and so can easily call one another using HTTP/REST or some RPC mechanism. However, a modern microservice-based application typically runs in a virtualized or containerized environments where the number of instances of a service and their locations changes dynamically.
Server-side service discovery: problem
How does the client of a service - the API gateway or another service - discover the location of a service instance?
Server-side service discovery: forces
Each instance of a service exposes a remote API such as HTTP/REST, or Thrift etc. at a particular location (host and port)
The number of services instances and their locations changes dynamically.
Virtual machines and containers are usually assigned dynamic IP addresses.
The number of services instances might vary dynamically. For example, an EC2 Autoscaling Group adjusts the number of instances based on load.
Server-side service discovery: solution
When making a request to a service, the client makes a request via a router (a.k.a load balancer) that runs at a well known location. The router queries a service registry, which might be built into the router, and forwards the request to an available service instance.
Server-side service discovery: example
An AWS Elastic Load Balancer (ELB) is an example of a server-side discovery router. A client makes HTTP(s) requests (or opens TCP connections) to the ELB, which load balances the traffic amongst a set of EC2 instances. An ELB can load balance either external traffic from the Internet or, when deployed in a VPC, load balance internal traffic. An ELB also functions as a Service Registry. EC2 instances are registered with the ELB either explicitly via an API call or automatically as part of an auto-scaling group.
Server-side service discovery: result benefits
- Compared to client-side discovery, the client code is simpler since it does not have to deal with discovery. Instead, a client simply makes a request to the router
- Some cloud environments provide this functionality, e.g. AWS Elastic Load Balancer
Server-side service discovery: result drawback
- Unless it’s part of the cloud environment, the router must is another system component that must be installed and configured. It will also need to be replicated for availability and capacity.
- The router must support the necessary communication protocols (e.g HTTP, gRPC, Thrift, etc) unless it is TCP-based router
- More network hops are required than when using Client Side Discovery
Server-side service discovery: related
- A router uses Service Registry
- A router might use a Circuit Breaker to invoke services
- Client Side Discovery is an alternative solution
Service registry: context
Clients of a service use either Client-side discovery or Server-side discovery to determine the location of a service instance to which to send requests.
Service registry: problem
How do clients of a service (in the case of Client-side discovery) and/or routers (in the case of Server-side discovery) know about the available instances of a service?
Service registry: forces
- Each instance of a service exposes a remote API such as HTTP/REST, or Thrift etc. at a particular location (host and port)
- The number of services instances and their locations changes dynamically. Virtual machines and containers are usually assigned a dynamic IP address. An EC2 Autoscaling Group, for example, adjusts the number of instances based on load.
Service registry: solution
Implement a service registry, which is a database of services, their instances and their locations. Service instances are registered with the service registry on startup and deregistered on shutdown. Client of the service and/or routers query the service registry to find the available instances of a service. A service registry might invoke a service instance’s health check API to verify that it is able to handle requests.
Service registry: example
The Microservices Example application is an example of an application that uses client-side service discovery. It is written in Scala and uses Spring Boot and Spring Cloud as the Microservice chassis. They provide various capabilities including a Netflix Eureka service registry.
Service registry: result benefits
Client of the service and/or routers can discover the location of service instances.
Service registry: result drawbacks
Unless the service registry is built in to the infrastructure, it is yet another infrastructure component that must be setup, configured and managed. Moreover, the service registry is a critical system component. Although clients should cache data provided by the service registry, if the service registry fails that data will eventually become out of date. Consequently, the service registry must be highly available.
Service registry: issues
You need to decide how service instances are registered with the service registry. There are two options:
- Self registration pattern - service instances register themselves.
- 3rd party registration pattern - a 3rd party registers the service instances with the service registry.
Service registry: related
- Client-side discovery and Server-side discovery create the need for a service registry
- Self registration and 3rd party registration are two different ways that service instances can be registered with the service registry
- Health Check API - the service registry invokes a service instance’s health check API to verify that it is able to handle requests
Self registration: context
You have applied either the Client-side Service Discovery pattern or the Server-side Service Discovery pattern. Service instances must be registered with the service registry on startup so that they can be discovered and unregistered on shutdown.
Self registration: problem
How are service instances registered with and unregistered from the service registry?
Self registration: forces
- Service instances must be registered with the service registry on startup and unregistered on shutdown
- Service instances that crash must be unregistered from the service registry
- Service instances that are running but incapable of handling requests must be unregistered from the service registry
Self registration: solution
A service instance is responsible for registering itself with the service registry. On startup the service instance registers itself (host and IP address) with the service registry and makes itself available for discovery. The client must typically periodically renew its registration so that the registry knows it is still alive. On shutdown, the service instance unregisters itself from the service registry.
This is typically handled by a Microservice chassis framework
Self registration: result benefit
A service instance knows its own state so can implement a state model that’s more complex than UP/DOWN, e.g. STARTING, AVAILABLE, …
Self registration: result drawback
- Couples the service to the Service Registry
- You must implement service registration logic in each programming language/framework that you use to write your services, e.g. NodeJS/JavaScript, Java/Scala, etc.
- A service instance that is running yet unable to handle requests will often lack the self-awareness to unregister itself from the service registry
Self registration: related
Service Registry - an essential part of service discovery
Client Side Discovery - one way that a service instance is discovered
Server Side Discovery - another way a service instance is discovered
Microservice chassis - self registration is the responsibility the microservice chassis framework
3rd Party Registration is an alternative solution
3rd Party Registration: context
You have applied either the Client-side Service Discovery pattern or the Server-side Service Discovery pattern. Service instances must be registered with the service registry on startup so that they can be discovered and unregistered on shutdown.
3rd Party Registration: problem
How are service instances registered with and unregistered from the service registry?
3rd Party Registration: forces
- Service instances must be registered with the service registry on startup and unregistered on shutdown
- Service instances that crash must be unregistered from the service registry
- Service instances that are running but incapable of handling requests must be unregistered from the service registry
3rd Party Registration: solution
A 3rd party registrar is responsible for registering and unregistering a service instance with the service registry. When the service instance starts up, the registrar registers the service instance with the service registry. When the service instance shuts downs, the registrar unregisters the service instance from the service registry.
3rd Party Registration: result benefits
The service code is less complex than when using the Self Registration pattern since its not responsible for registering itself
The registrar can perform health checks on a service instance and register/unregister the instance based the health check
3rd Party Registration: result drawback
The 3rd party registrar might only have superficial knowledge of the state of the service instance, e.g. RUNNING or NOT RUNNING and so might not know whether it can handle requests. However, as mentioned above some registrars such as Netflix Prana perform a health check in order to determine the availability of the service instance.
Unless the registrar is part of the infrastructure it’s another component that must be installed, configured and maintained. Also, since it’s a critical system component it needs to be highly available.
3rd Party Registration: related
Service Registry
Client Side Discovery
Server Side Discovery
Self Registration is an alternative solution
