Internet-Wide Measurements

Question 1

Why do we do internet wide measurements?

Answer 1

to evaluate Properties of a service deployed on the public internet.

Question 2

Which ethical problems do arise from measurement?

Answer 2

• Creates additional traffic
• Creates load on routers and hosts
• Might uncover personal information
• Might be intrusive

Question 3

What ethical considerations have to be made in order to do ethical scanning?

Answer 3

• Scan with a moderate rate
• Distribute the load as good as possible
• Do not publish data without anonymization or limited access
• Inform about the scanning behavior and react to complaints

Question 4

Which operation modes does NMap provide?

Answer 4

• Host discovery
• Service detection
• OS detection
• Execution of custom scripts

Question 5

What scans does NMAP provide?

Answer 5

• TCP raw socket scans
• TCP connect scans
• ICMP Ping scans
• UDP Payload scans

Question 6

How can NMAP scan via TCP Raw sockets?

Answer 6

Syn: find open ports
NULL/Fin/Xmas (Fin/Psh/Urg): Rst if closed, nothing if open
ACK: Determine if port is filtered or unfiltered
Window: Filtered/unfiltered - if Window > 0 in Rst -> open
Maimon: Fin+Ack -> Rst on open/closed

Question 7

What is the problem with using nmap for internet wide scanning?

Answer 7

It performs stateful scanning and therefore takes a very long time

Question 8

How does ZMap perform on internet-wide scans?

Answer 8

It uses stateless scanning (therefore no timeout detection).
AES encrypted IDs as sequence numbers.
Validation seq -1

Question 9

What are Hitlists?

Answer 9

A list of addresses, most likely responsive, of feasible size.

Question 10

Why cant we use classical hitlist mehtods for IPv6?

How can we create them?

Answer 10

Address space is too large

• List of Addresses (raw packet traces, traceroute, flow data from measurement point)
• List of Domains (Unranked, Ranked), (DNS zones, certificate transparency)
• Active Scans (rDNS walking)
• Machine Learning

Question 11

How are Alexa, Majestic Million and Umbrella hitlists created?

Answer 11

Alexa: volunteers via toolbar
Majestic: Web crawler searches for incoming/outgoing links
Umbrella: Based on DNS requests

Question 12

How does rDNS walking work?

Answer 12

1. Start at root ip6.arpa.
2. Query nibble values
3. Descend into subtree
4. Check next nibble

Question 13

Why should we treat top lists carefully?

Answer 13

Frequent changes over time [12]
• Weekend effect [12, 13]
• Different user behavior changes lists on the weekend
• Focus towards entertainment and streaming on the weekend
• Clustering Effect [13]
• Large clusters with same rank
• Ordered alphabetically
• Size is not always 1 Million

Question 14

How does the IPv6 hitlist work on the chair?

Answer 14

Filters aliased prefixes and applies blacklists

• Not globally routed
• Blacklisting requests
• Not responsive for 30 consecutive days.

Tests reachability daily
• ICMPv6
• TCP/80 (HTTP)
• TCP/443 (HTTPS)
• UDP/53 (DNS)
• UDP/443 (QUIC)
• Uses ZMapv6