VA Session 7 ^

Question 1

Subsetting & Slicing methods with pandas

Answer 1

• loc
• iloc

Question 2

df.loc[]

Answer 2

Access rows or columns by labels or Boolean array; slices: start & stop included

Question 3

df.loc[]:
- elements
- row with label 1 as series
- row with label 1 as Data Frame
- rows from start (0) to end (5)
- all rows & named column

Answer 3

• df.loc[“row_name”, “column_name”]
• df.loc[“A”]
• df.loc[[“A”]]
• df. loc[“A”:”C”]
• df.loc[:,“Nr-in_Alphabet”]

Question 4

df.iloc[]

Answer 4

• Access rows or columns by their integer position or with Boolean array (position go from 0 to length -1)
• slice: end number not included
• faster for selecting rows than loc

Question 5

• df.iloc[]
• elements
• row at position 1 as series
• row at position 1 as Data Frame
• rows from index (0) to (4)
• all rows & first two columns

Answer 5

• df.iloc[row_index, colum_index]
• df.iloc[1]
• df.iloc[[1]]
• df.iloc[0:5]
• df.iloc[:,0:2]

Question 6

Subsetting rows with index 0 to 4

Answer 6

df[0:5]

Question 7

Subset columns

Answer 7

• df[“column_name”]
• df[[“col_name1”, “col_name2”]]

Question 8

Subset rows & columns

Answer 8

df[0:5] [[“col_name1”, “col_name2”]]

Question 9

Subsetting vs referencing of datasets

Answer 9

• Copying: df.copy() -> Create new true copy of Data Frame
• Referencing: subsetting & storing results in new Data Frame -> new one still referencing the original DataFrame: when changing original data, also copied would be changed

Question 10

Filtering data: Subset Dataframe’s rows or columns according to specified row or column labels

Answer 10

df.filter(like=”culmen”, axis = 1)

Question 11

Cleaning data - Checking & removing duplicates

Answer 11

• Duplicates: already exist in stored data or created when merging datasets
• df.duplicated(): Check duplicates row-wise
• df.nunique(): Check duplicates column-wise
• df.drop_duplicates(): Remove duplicate rows

Question 12

Cleaning data - Remapping values (e.g. due to faulty data or analysis requires transformation)

Answer 12

df.replace()

Question 13

Cleaning data - Dealing with text

Answer 13

• Common issues in text data quality -> more categories than actually exist
• e.g. “Copenhagen”, “COPENHAGEN”, “Copenhagen
• df[“column”].str.strip(): Remove spaces
• df[“column”].str.upper()
• df[“column”].str.upper(): Change capitalization

Question 14

Reshaping data - Unpivot DataFrame from wide to long format

Answer 14

melt()
wide_to_long()

Question 15

Reshaping data - Change from long to wide format

Answer 15

pivot()

Question 16

JavaScript Object Notation (JSON)

Answer 16

• common format for strong data as human-readable text (e.g. used by many web applications & servers)
• pd.read_json(“/path/file.json”): Read a JSON file
• pd.to_json(“/path/file.json”): Write a JSON file

Question 17

Joining data by join

Answer 17

• pd.merge(df1, df2, how =”inner”, left_on=”id”, right_on=”id”)
• Join datasets, can also be done on row indexes rather than specific id columns

Question 18

Joining data by union

Answer 18

• concat()
• datasets that have same columns

Question 19

Joining data by intersections

Answer 19

merge()

Question 20

Df[…] vs loc & iloc

Answer 20

• [] + index -> selects rows
• [] + Names -> selects rows and columns
• df[index] [names] -> selects rows and columns