5. Networking Services (I) Flashcards
Internet Protocol - IP
Acts as a computer “address” in the internet network, such that other PCs know where to send the information to.
Internal IPs + DNS
Internal IPs use DNS to translate all the different IP ranges into websites.
DNS types:
1. Zonal - recommended due to higher reliability
2. Global
- Each instance has a hostname (so like ‘printer-work’ rather than printer-work.us-central1-a.c.my-project-id.internal - FQDN (Fully Qualified Domain Name)
- For instance to be able to identify what’s its external IP, it creates like a lookup table using DHCP (Dynamic Host Configuration Protocol) - auto assignment of IPs rather than manual
Note: you can create alias IP ranges (so if the VM was assigned an IP, the services on the VM can have sub-IPs)
Note: having internal DNS for the instances allows us to ping the instances using their names rather than the internal IPs, the internal IPs may change, but the name won’t, so DNS helps us to have something ‘fixed’ we can refer to.
Note: it’s good to have instances in the same subnet because firewall rules can be then simply applied to that subnet rather than each instance individually.
- it’s RECOMMENDED to have just internal IP, not external, for the instances that need to communicate with the outside inet for updates, can use Cloud NAT or Private Google Access to access Google APIs and Services without external IPs.
Commands to create a new VPC network, make it custom and add subnets
gcloud compute networks create VPC-name –subnet-mode=custom
Adding firewall rules
-tcp: 22 (SSH) , 3389 (RDP)
gcloud compute firewall-rules create my-firewall-name –direction=INGRESS –priority=1000 –network=VPC-name –action=ALLOW –rules=tcp:22,tcp:3389,icmp –source-ranges=0.0.0.0/0
Pricing of ingress/egress traffic of VMs
- all ingress is FREE
- egress in the same zone and to internal IP/Google cloud service (youtube etc) is FREE
- egree to other zones/regions/external IPs is CHARGED
Lab: enable Private Google Access & Cloud NAT on instances without external IP
- if a VM instance has no External IP, we won’t be able to SSH into it, so use IAP (Identity-Aware Proxy) tunnel from the Cloud Shell to get the secure connection.
gcloud compute ssh vm-name –zone=us-central1-c –tunnel-through-iap
- when a VM has only internal IP, other instances within the same network can only reach it using a managed VPN gateway or Cloud IP tunnel
- Google Private Access is enabled at the subnet level, so to enable it, need to go to the VPCs and find our network, click on the subnet, edit, and then tick the Google Private access option.
To enable Cloud NAT:
- Go to Network Services –> Cloud NAT –> select my VPC, the region.
-Then need to create Cloud Router
Open Systems Interconnection Model (OSIM)
7 layer OSIM model:
- application (HTTP - HTTPS - DHCP - DNS - SSH - Telnet)
- presentation
- session
- transport (TCP - UDP , IP Packets)
- network (IPv4 - IPv6 , IP address Subnets)
- data link
- physical
Each layer has a protocol.
Internet Protocol: Network layer
IPv4 (came out in 1981) - 4 Bn possible combinations
IPv6 (came out in 2017) to tackle the problem of the amount of usable IPs being slowly used up by IPv4
Internet Protocol: Network layer
IPv4 notation (dotted decimal notation)
Each number ranges from 0 to 255
Each group contains 8 bits, so 1 byte
192.168.255.255
Private IP addresses were standardised by standard RFC1918
Internet Protocol: Network layer
IPv4: Classless Inter-Domain Routing (CIDR)
192.168.0.0/16
It means we are “sub-netting” the network
The bigger the /.. part, the smaller the network.
Internet Protocol: Network layer
IPv4: Classless Inter-Domain Routing (CIDR):
Helpful References
Note: bold parts are FIXED
192.168.0.0/8 - 16+ million IP addresses
192.168.0.0/16 - 65,536 IP addresses
192.168.0.0/24 - 256 IP addresses
192.168.1.2/32 - 1 IP address (good for your personal VPN network configuration)
0.0.0.0/0 - All IP addresses
Internet Protocol: Network layer
IPv6: Hexadecimal notation
Each grouping is a Hextet = 16 bits
1452:0db8:0000:0000:0000:fe02:0042:8452
1452:0db8:0:0:0:fe02:0042:8452 (same as above)
1452:0db8::fe02:0042:8452 (same as above)
The whole address is 128 bits
::/0 - All IP addresses
Internet Protocol: Transport layer (TCP/UDP)
IP Packet - unit of info
TCP - transmission control protocol
UDP - user datagram protocol
A typical IP Packet / datagram includes:
- Source IP address
- Destination IP address
- Protocol Port Number (source/destination)
- Data
Internet Protocol: Application layer (HTTP - HTTPS - DHCP - DNS - SSH - Telnet)
Used by applications that use the inet.
HTTP/HTTPS - used for Load Balancing
DHCP
DNS - uses UDP transport layer on port 53
SSH - on port 22
Telnet
Virtual Private Cloud (VPC)
- global resource
- virtualised Datacentre
- VPCs live within projects
- VPCs don’t have any IP address ranges associated with them
- resources within a VPC can communicate with one another using internal (private) IPv4 addresses
- IPv4 only
- VPC contains a default network
- 2 network types: Auto Mode or Custom Mode (no automatic creation of subnets or IP ranges)
Custom mode is RECOMMENDED
Different Networks = different VPCs (external IP required)
VPC peering or a VPN connection is required for the VMs from different networks to communicate with each other, even if they are located in the same zone
VPC default network
Within a default network:
- IPs with subnet in each region is created
VPC default network:
Command to change subnet
gcloud compute networks subnets expand-ip-range default –region=us-west1 –prefix-length=16
VPC Network Subnets
- the name or region of a subnet cannot be changed after it is created
Note: Google allows to increase the subnet IP space without any shut down time
Note: the addresses within THE SAME VPC NETWORK cannot overlap + they must be inside the RFC1918 address-space
Auto Mode network starts with /20 network range and can be expanded to /16
Note: network range can be expanded but cannot be reduced!
Note: There are some reserved IP addresses that I cannot choose (1st address - reserved for Network, 2nd - for Default Gateway, 2nd-to-last - for Google Cloud future use, last - for broadcast)
Routing (definition)
Routing - defines the network traffic path from a VM instance to other destinations
Routing: Types
System-generated
- default
- subnet route
Custom routes
- static routes
- dynamic routes
Routing: default & subnet type routes
Default route
- path to the inet
- path for Private Google Access
- destination covers all IP addresses, so it has the lowest priority
- can only be replaced with a custom route
Subnet route
- define paths to each subnet in the VPC
Routing: static & dynamic type routes
Static route
- created automatically when creating Cloud VPN tunnels
- so if at some point we will have more subnets, Cloud VPN won’t know it, we will need to go in and manually configure this
Dynamic route
- manage routes between a VPC and on-premises networks
- IP ranges include numbers outside the VPC network
- Used with VPN and Interconnect
Private Google Access (general info)
VM instances that only have the internal IP addresses use Private Google Access.
This allows them to reach the external IP addresses of Google’s APIs and services.
Private Google Access can be applied to one subnet within the same network but for example not the other. (So one region will have the Private Google Access, but won’t have the external IP address, while another region will have an external IP address and so will not require the Private Google Access)
Private Google Access ( ways to connect)
- Private Google Access for on-premisses hosts (through Cloud VPN tunnel or Cloud Interconnect)
- Private Services Access (through a VPC network peering connection)
- Serverless VPC Access (internal VPC connection)
IP Addressing (Decision Tree: 1st choice)
(Internal - Private / External - Public)
IP Addressing (Decision Tree: Internal IP addresses 2nd + 3rd choice)
Alias IP
Auto
- automatic subnet selection
Custom
- specify which subnet each IP comes from
Ephemeral IP
- IP address that does not persist beyond the life of the resource
Static IP
- remain with the project until you remove the project
- must specify which subnet the IP address comes from
- can be specified when we are creating a VM instance, from the drop-down menues
IP Addressing (Decision Tree: External IP addresses 2nd choice)
Ephemeral
Static
- available as a regional or global resource
- More expensive to have if the static IP is not assigned to any resource eg VM instance or a Forwarding Rule.
- Can be reserved only if you use the /24 mask or greater IP range.
Internal IP address reservation
- can reserve an internal IP address for a specific resource usage, so even if it is deleted, the resource still has the IP
- can reserve a specific address and then associate it with a specific resource
- can specify an ephemeral internal IP address for a resource and then promote the address
External IP address reservation
Can reserve:
Regional IP address
- for regional Load Balancers
Global IP address
- for global Load Balancers
CLI: command to check the IP addresses in a specific project for all the VM instances
gcloud compute addresses list
Creating an External static IP address
VPC network –> External IP addresses –> create
IPv6 External Static IP address: use case
Can only be used by Global Load Balancing
Global Load Balancing use premium network service tier and so will be more expensive.
Unused Static IP addresses are also billed!
Delete an Internal IP address (using CLI because impossible to do through the Console)
gcloud compute addresses delete internal-address-name –region us-east1
VPC firewalls rules
Items mentioned when creating a firewall rule:
-protocol
- ports
- sources
- destination
Implied Rules
allow egress
deny ingress
Some important default firewall rules
default-allow-icmp:
default-allow-internal:
default-allow-rdp : on port 3389 (windows remote desktop protocol)
default-allow-ssh: on port 22
Note: they all have 2nd to lowest priority.
Note: firewall rules only support IPv4.
Note: firewall rules apply to separate instances but they are enforced at the VPC level!!
Firewall rule components
Network: which VPC network the rule applies to
Priority
Direction of traffic: Ingress/Egress
Action on match: Allow/Deny (whether the rule permits or blocks the connection)
Target: here we specify which instances within the VPC the rule applies to; can choose to apply to all instances within the VPC OR to apply to instances that are tagged as ___ OR to apply to instances that have a service account.
Source Filter: exists only for Ingress rules, specify source IP, tags or source service account
Protocols and Ports: if omitted, the firewall rule will be applied to the traffic from all protocols and all ports
Note: “Disable rule” button exists to turn ON or OFF the rule
How to make the private network communicate with Cloud Storage without external IP?
Use Private Google Access: in the VPC network details for the private network –> edit –> turn the Private Google Access on.
Large DEMO steps
- Enable Compute Instance API.
- Create a custom VPC network, not default (name, description, subnet: public – region, IP range 10.0.0.0/24, private – 10.0.5.0/24; region choices for private and public must be different
- DNS API must be enabled here.
- Create a bucket in Cloud Storage and upload some files.
- Create Compute Engine Instances: public and private; add label (key - value); change region to match the public network region
- For service account set up the correct permissions: Set access for each API - for public give permission of Read Write to Compute Engine (it will be allowed to SSH into the private instance) and Storage; for private give permission of Read Write to Storage only.
- Create a network tag below in the Networking section, and choose the custom network here. For private subnet remove the external IP.
Create Firewall rule
For Public instance we want to create for it a public access & SSH access: name, network - custom; priority; direction of traffic (ingress); targets - specify tags; source IP range 0.0.0.0/0; for protocols and ports click on TCP on port 22 AND Other protocols put icmp (to ping the private instance).
For Private instance we want to create a rule to access this private instance from a public one: name, network - custom; priority; direction of traffic (ingress); targets - specify tags; source IP range 10.0.0.0/24; for protocols and ports click on TCP on port 22 AND Other protocols put icmp (to ping the instance).
Note: to test if we can SSH into the private instance from the public instance on the CLI: gcloud compute ssh –project project-name –zone us-east4-c private-VM-instance-name –internal-ip
TPU (Tensor Processing Unit)
Better than GPU/CPU because it’s designed for training AI models
VM lifecycle
- Resource provisioning (instances.start())
- Staging - resources are gathered and the instance is being prepared for the launch
- Running (at the start it enables SSH/RDP), at this stage, we can change the zone of the VM, change boot disk. All this without stopping the instance
- Stop the instance when: adding more CPU/delete VM
- Suspend instance when: resume or delete instance
- When maintenance needs to be done, a host can be used to temporary live migrate the VM into a different place. Or the VM can simply be stopped for that time. Set these in the settings.
- Use Patch management to manage patches to the OS of the VMs all at the same time.
OS patch management:
- Patch compliance reporting - reports the patch state of the VMs (insights + recommendations)
- Patch deployment - automation of the patches and updates to the OS processes
Note: you can select which patches to apply, what time, apply advanced patch configuration settings (like add script to say what to do before/after a patch), manage patches from a centralised location
Settings that cannot be changed when the VM is running (but can be when it’s stopped)
- CPUs
Can change
- tags
- On host maintenance (eg migrate) + automatic restart(on) (both recommended when it comes to maintenance)
- sometimes it’s best to just stop the instance while the maintenance is happening because the migration will change its MAC address
Windows VM
- Instead of SSH will now see RDP
- Need to create a password first
Shielded VM
Turn on vTPM –> check the integrity of the syst. Check what were the software configs before (last login) vs current login
Sum up of VPNs
Accessing Internal IP
1. Dynamic routing with Cloud Router
- Routing allows to see what are the routes available on the other side (VLAN - on-prem needs to know how many/which subnets - on cloud, are available)
- BGP is needed to set up dynamic routing = Cloud Router
- the speed/throughput depends on traffic/inet provider, might be slow
- if cost is priority => use Cloud VPN
- if reliability + security is priority => use dedicated interconnect
-
Cloud Interconnect (dedicated interconnect) & Peering
- min 10GB –> up to 100GB/s
- get this option only if you need at least 10GB/s speed -
Partner Interconnect
- if reliability (throughput) + cost is priority (don’t need 10GB/s) OR you are not close to the colocation facility => use partner interconnect
- less costs
- reliable
- Provides L2 (you set up your own BGP routing) or L3 (the partner manages the Cloud Routing = BGP etc) options
Public IP connection to Google products/services from an Internal IP
1. Direct Peering
- when you use Google services (gmail, YouTube, GDrive…) but you don’t want the traffic to go through the public connection, so you create a private connection between Google and yourself
- must be able to reach Google at PoP
- min 10GB
-
Carrier Peering
- use when you have no access to the PoP locations
————————-
VPC to VPC connection -
Shared VPC
- one project becomes the host project. All other projects will have access to this host
- projects have to all be within your organisation -
VPC Peering
- can be inter-organisational
- both admins of VPCs need to agree to do the connection
