Lecture 14 Revision: Log File Analysis Flashcards
What format are Apache Log Files stored in?
CLF (Common Log Format)
What is the structure of a Common Log File entry?
%h %I %u %t %r %s %b
%h = IP address of remote host
%I = RFC 1413 identity (host name in human readable form) - if no value a hyphen is used. Often no value
%u = user identification (the user logged into the apache server at that point. replaced with a hyphen if no value filled. Often no value.
%t = time request was recieved (strict format of 23/Mar/2008:00:03:10 +0000 with the + at the end being time diff from UTC)
%r = request line from client (i.e the GET request e.g. GET /javascript/email.js HTTP/1.1 - remember there is a GEWT request for each peice of data on the webpage)
%s = server status code (e.g. 200 = ok, 204 = error if file does not exist on the server - the codes are built into Apache)
%b = size of the object returned in bytes.
What is the Combined Log Format?
A combined log format is the same as the common log format but with additional info at the end.
The info at the end is:
%ref
%useragent
%ref = the HTTP request header(i.e. what you see in the address bar - the url)
%useragent = the user agent HTTP request header (the ‘user agent string’ unique string for that browser that made the request)
What is a named capturing group?
A named capturing group is a feature in regex that allows you to assign a name to a specific matched group.
Instead of accessing the group by its numeric index (like group(1)), you can reference it by its name, which makes your code more readable and easier to maintain.
To define a named capturing group, you use the syntax (?P<name>pattern):</name>
name: The name you want to assign to the group. It must be a valid Python identifier (letters, numbers, and underscores, starting with a letter).
pattern: The regex pattern you want to capture.
What is the concept of the datetime library?
Manipulating dates and times in strings is very messy.
The built-in Python datetime library simplifies working with dates & times.
The concept is that:
- Dates and times in any format in a string can be read into a datetime object.
- Comparing datetime objects using logical expressions gives the correct chronological results.
- A datetime object can be output as a string in any format
How do you access the in built datetime library in Python?
from datetime import datetime
What are POSITIONAL ARGUMENTS?
These are arguments passed to a function in the exact order in which they are defined in the function signature.
The position in the function call determines which parameter gets which value.
What are NAMED ARGUMENTS?
Named (Keyword) Arguments
These are arguments that are explicitly passed with a name (or keyword) in the function call.
You specify the parameter name followed by its value, regardless of the order.
Named arguments are particularly useful when working with functions that have parameters with default values
What are the key differences of POSITIONAL and NAMED arguments?
Key Differences
Order Sensitivity:
Positional Arguments: The order in which arguments are passed matters.
Named Arguments: Order doesn’t matter; parameter names are explicitly specified.
Readability:
Named arguments can make function calls more readable, especially when there are many parameters or default values involved.
Mixing Positional and Named Arguments:
You can mix both types, but positional arguments must always come before named arguments in the function call.
What does the method strptime() (string parse time) do?
The method strptime() (string parse time) converts a string containing a date and time into a datetime object according to a format string.
What function can we import to help us sort data?
from operator import itemgetter
(this imports the function itemgetter from the module called operator.
itemgetter is a function from the operator module that is used to retrieve specific items (by their index or key) from objects like lists, tuples, or dictionaries. It creates a callable object that fetches the specified item(s) when invoked.
What does the function sorted() do?
The sorted() function in Python is used to return a new sorted list from an iterable (such as a list, tuple, string, or dictionary).
It does not modify the original data structure but instead produces a sorted copy, making it particularly useful when you want to preserve the original order of the data.
By default, sorted() sorts elements in ascending order.
You can sort in descending order by using the reverse parameter.
The key parameter allows you to define a custom sorting rule by providing a function or callable.
What is a tuple?
In Python, a tuple is a collection of ordered, immutable elements. It is similar to a list, but unlike lists, tuples cannot be changed after they are created. Tuples are often used when you want to group data together but ensure that the data remains constant.
What are the key features of a tuple?
Ordered:
The items in a tuple are stored in a specific sequence, and their order will not change.
You can access elements using their index, starting from 0.
Immutable:
Once a tuple is created, its elements cannot be modified (no adding, removing, or updating elements).
This immutability makes tuples hashable, so they can be used as keys in dictionaries or added to sets.
Allow Duplicate Elements:
Like lists, tuples can have multiple occurrences of the same element.
Can Store Mixed Data Types:
A tuple can contain elements of different data types (e.g., integers, strings, floats, etc.).
Defined Using Parentheses:
Tuples are typically created using parentheses (). So the tuple content will be stored within () and each item held in the tuple is seperated by a comma
