Pandas

Question 1

DataFrame:

Create a dataframe from scratch

Answer 1

First create lists
age = [25,20,26,30]
height = [150,160,180,170]
names = ['Mārtiņš','Aiga','Kristiāns','Valters']
column_names = ['Age','Height','Names']

list_cols = [name, age, height]

zipped = list(zip(column_names,list_cols))

data = dict(zipped)

df = pd.DataFrame(data)

Question 2

DataFrame:

Add a new column with values 0

Answer 2

df[‘Salary’] = 0

Question 3

DataFrame:

Rename columns

Answer 3

df.columns = [‘name’,’school’,age’]

Question 4

DataFrame:

Rename indexes

Answer 4

df.index = [‘A’, ‘B’, ‘C’]

Question 5

DataFrame:

Create a dictionary from lists

Answer 5

list1 = [  ]
list2 = [  ]
column_names = [  ]

columns = [list1, list2]

created_tuples = list(zip(column_names,columns]

created_dict = dict(created_tuples)

Question 6

Get how many rows and columns the DataFrame has

Answer 6

my_dataframe.shape

Question 7

Get the column names

Answer 7

my_dataframe.columns

Question 8

DataFrame: 
Slice the dataframe.
1) first 5 rows
2) last 5 rows
3) columns 3 to 5 included
4) each 3rd row

Answer 8

my_data.iloc[:6,:]
my_data.iloc[-5:,:]
my_data.iloc[:,3:6]
my_data.iloc[::3,:]

Question 9

DataFrame:

see the first 10 rows quickly

Answer 9

my_data.head(10)

Question 10

DataFrame:

see the last 8 rows quickly

Answer 10

my_data.tail(8)

Question 11

DataFrame:

see the column names, their types and count quickly

Answer 11

my_data.info()

Question 12

DataFrame:

assign a value to some element in DataFrame

Answer 12

my_data.iloc[5,10] = 29

Question 13

DataFrame:
assign NaN to every 3rd row in the last column.
Which rows will be affected?

Answer 13

import numpy as np
my_data.iloc[::3,-1] = np.nan

nan
unchanged
unchanged
nan
unchanged
unchanged
nan
...

Question 14

DataFrame:

transform DataFrame to numpy array

Answer 14

my_data.values

Question 15

DataFrame:

get the index column

Answer 15

my_data.index

Question 16

DataFrame:

Create a new column with some values

Answer 16

df[‘new_col’] = 0

Question 17

DataFrame:

Assign new names for the columns

Answer 17

df.columns = [‘name’,’surname’,’age’]

Question 18

DataFrame: 
Fully define pandas  csv import with 
1) custom column names, 
2) what multiple values indicate invalid data, 
3) what symbol separates values, 
4) transform a column to datetime,
5) how not to show first few rows.

Answer 18

path = /folder/file.csv
col_names = [‘name’, ‘surname’, ‘age’]
pd.read_csv(path, header = none, names = col_names, na_values = ‘-1’)

Question 19

DataFrame:

If csv has year, month, day in separate columns, how to read_csv so tha it combines the 3 columns in one.

Answer 19

pd.read_csv(path, parse_dates = [[0 , 1 , 2]] )

Question 20

DataFrame:

Get rid of one column in dataframe. How would you do it?

Answer 20

If the df has more columns than needed, define meaningul cols names

meaningful_columns = [‘name’, ‘surname’]

assign the columns to itself

df = df[meaningful_columns]

Question 21

DataFrame:

Export the the dataframe to csv or excel

Answer 21

path = ‘my_file.csv’

df.to_csv(path)

path2 = ‘my_file2.xlsx

df.to_excel(path2)

Question 22

DataFrame:

If data has a column with dates, that you want to have as index how would you import that csv

Answer 22

pd.read_csv(path, index_col = ‘dates’, parse_dates = True)

Question 23

DataFrame:

Have the name of the plot lines on the plot.

Answer 23

df[‘open’].plot(legend = True)
df[‘close’].plot(legend = True)
plt.show()

Question 24

DataFrame:

Plot specific columns (not the index).

Answer 24

df. plot( x = ‘Month’, y = [‘salary’, ‘overhead’]

plt. show()

Question 25

DataFrame:

Create a scatter plot with different sizes of dots.

Answer 25

df. plot(kind = ‘scatter’, x = ‘year’, y = ‘age’, s = df[‘size’])
plt. show()

Question 26

DataFrame:

Plot two columns separately. Create a box plot.

Answer 26

df[‘first’, ‘second’].plot(kind = ‘box’, subplots = True)

plt.show()

Question 27

DataFrame:
Create a CDF and PDF plots in two rows. Scale the vertical axis. Change the horizontal division - make it finer. State the horizontal from … to… values.

Answer 27

fig, axes = plt.subplots(nrows=2, ncols=1)

Plot the PDF

df. fraction.plot(ax=axes[0], kind=’hist’, normed = True, bins = 30, range=(0,.3))
plt. show()

Plot the CDF

df. fraction.plot(ax = axes[1], kind = ‘hist’, normed=True, cumulative = True, bins = 30, range=(0,.3))
plt. show()

Question 28

DataFrame:

Get the statistical information about the dataset.

Answer 28

df.describe()

Question 29

DataFrame:

How many non nul entries in a DataFrame column?

Answer 29

df[‘my_col’].count()

Question 30

DataFrame:

Get the mean of a DataFrame column. What does it do to null values?

Answer 30

df[‘my_col’].mean()

Question 31

DataFrame:

Get the standart deviation of a DataFrame column. What does it do to null values?

Answer 31

df[‘my_col’].std()

Question 32

DataFrame:

What is a median? How to get it for a DataFrame column?

Answer 32

df[‘my_col’].median()

Question 33

DataFrame: 
First quartile : 25%
Second quartile : ....?
Third quartile : ....?
Get the value for 20 %  percentile.

Answer 33

df.quantile(0.2)

Question 34

DataFrame:

Get the interquantile range between 0.2 and 0.8

Answer 34

df.quantile([0.2, 0.8])

Question 35

DataFrame:

Get the minimum and maximum values.

Answer 35

df. min()

df. max()

Question 36

DataFrame:
Get the mean value calculated over
1) each row
2) each column

Answer 36

#mean calculated for each row
df.mean(axis = 1)

#mean calculated for each column
df.mean(axis = 0)

Question 37

DataFrame:

What information can you get with df[‘species’].describe() Species column has categorical values.

Answer 37

df['species'].describe()
give 
count
top
unique
frequency - occurrences of top

Question 38

DataFrame:

Get list of values in a column without repeating.

Answer 38

df[‘species’].unique()

Question 39

DataFrame:

Add transparency to a plot

Answer 39

df. plot(alpha = 0.8)

plt. show()

Question 40

DataFrame:
Create two separate DataFrames from one by subsetting- one DF has rows about ‘Dog’ and the other one about ‘Cat’. Main DF has a column animal.

Answer 40

dog_indices = df['Animal'] == 'Dog'
cat_indices = df['Animal'] == 'Cat'
dog_df = df.loc[dog_indice]
cat_df = df.loc[cat_indice]

Question 41

DataFrame:
Df has a index column with datetime.
1) Get rows in a specific datetime range from a specific column.
2) Get rows on a specific day from a specific column.

Answer 41

df. loc[‘2019-02-25’:’2019-03-25’, ‘product’]

df. loc[‘2019-02-25’, ‘product’]

Question 42

DataFrame:
Df has a index column with datetime.
1) Get rows in a specific datetime range from a specific column.
2) Get rows on a specific day from a specific column.

Answer 42

df. loc[‘2019-02-25’:’2019-03-25’, ‘product’]

df. loc[‘2019-02-25’, ‘product’]

Question 43

DataFrame:

Transform a list of datetime strings to df datetime

Answer 43

string = [‘2019-02-25’,’2019-03-25’]

pd.to_datetime(string)

Question 44

DataFrame:
Have a dataframe df1 adapt to a new datetime index from another dataframe df2.

What will be the result if df1 doesn’t have data on a dates in df2?

Answer 44

df.reindex(df2.index)

Question 45

DataFrame:
Have a dataframe df1 adapt to a new datetime index from another dataframe df2. If df1 doesn’t have data on all df2 dates, have the empty places be the nearest preceeding entry in df1.

Answer 45

df.reindex(df2.index, method = ‘ffill’)

Question 46

DataFrame:
Have a dataframe df1 adapt to a new datetime index from another dataframe df2. If df1 doesn’t have data on all df2 dates, have the empty places be the nearest next entry in df1.

Answer 46

df.reindex(df2.index, method = ‘bfill’)

Question 47

DataFrame:
List contains this:
time_list = [‘20100101 00:00’, ‘20100101 01:00’, ‘20100101 02:00’]

How to transform it to datetime?

Answer 47

pd.to_datetime(time_list, format = ‘%Y-%m-%d %H:%M’)

Question 48

DataFrame:
We have a dataframe df1 that has hourly entries.

Create a df2 that show daily mean, sum, max, etc.
How to get second, minute, hourly, daily, weekly, monthly, yearly calculations?

How do you get calculations over say 3 hour periods?

Answer 48

df2 = df1.resample(‘D’).mean()

Seconds S
mins T or min
hours H
days D
weeks W
Months M
Yearly A

To get 3 hour periods use 3H.

Question 49

DataFrame:

How to get the maximum value of mean values in a month if the df has daily entries?

Answer 49

df.resample(‘M’).mean().max()

Question 50

DataFrame:

Get the rolling mean values over defined period of time.

Answer 50

df.rolling(window = 24).mean()

Question 51

DataFrame:

read_csv and specify which column should be transformed to datetime objects.

Answer 51

pd.read_csv(path, parse_dates = [‘my dates’])

Question 52

DataFrame:

Create a df series that has it’s text in upper or lower case.

Answer 52

df[‘company’].str.upper()

Question 53

DataFrame:

Create a df series that has True False for rows that contain a word ‘ware’.

Answer 53

df[‘company’].str.contains(‘ware’)

Question 54

DataFrame:

Count the number of rows that has the letter combination ‘ware’ in a column.

Answer 54

df[‘product’].str.contrains(‘ware’).sum()

the first par return series of True / False. True + True = 2 etc.

Question 55

DataFrame:

Extract the value of hours from df datetime column.

Answer 55

df[‘my date’].dt.hour

Question 56

DataFrame:
df has the population of the world for every decade. Upsample the df to every year and fill in the blanks with linearly changing values.

Answer 56

df.resample(‘A’).first().interpolate(‘linear’)

Question 57

DataFrame:

Strip the whitespaces from a df header.

Answer 57

df.columns = df.columns.str.strip()

Question 58

DataFrame:

Two columns contain date and time. Create series to combine both and have the type of series as Datetime.

Answer 58

pd.to_datetime( la[‘Date (MM/DD/YYYY)’] + ‘ ‘ + la[Wheels-off Time’] )

Question 59

DataFrame:
How to define line style in plots?
What are the options?

Answer 59

df.plot(style = ‘k.-‘)

colors:
k - black
b - blue
c - cyan
r - red

marker type:
. dots
o circles
s squares
* stars

line type:
: dotted
- solid

Question 60

DataFrame:

Have everything bellow a plot line in a color.

Answer 60

df.plot(kind = ‘area’)

Question 61

DataFrame:
You have a df with columns:
index date temperature

make the date column become the index column.

Answer 61

df.set_index(‘date’, inplace = True)

Question 62

DataFrame:
You have a string, that contains all names for the columns.

String looks like this: cols = ‘Wban,date,Time,StationType,sky_condition,sky_conditionFlag’

How would you get this become the header of a datafram?

Answer 62

df_cols = cols.split(',')
df.columns = df_cols

Question 63

DataFrame:

You have a list of column names that you want to remove from a df. How to do it?

Answer 63

df_dropped = df.drop(list_to_drop, axis = 1)

Question 64

DataFrame:

Convert a column to a different data type - a string

Answer 64

df_dropped[‘date’] = df_dropped[‘date’].astype(str)

Question 65

DataFrame:

If you want to have df indexes to be regular 1, 2 , 3 … etc.

Answer 65

my_series.reset_index()

Question 66

DataFrame:

Find if there is a relationship between columns!

Answer 66

df.corr()

Question 67

DataFrame:

extract df2 from df1 such that df2 contains only the rows where df1 column’s values are above, say, 10.

Answer 67

df2 = df1.loc[df1[‘column name’] > 10]

Question 68

Create a DataFrame with 3 columns:

names = [‘United States’, ‘Australia’, ‘Japan’, ‘India’, ‘Russia’, ‘Morocco’, ‘Egypt’]
dr = [True, False, False, False, True, True, True]
cpc = [809, 731, 588, 18, 200, 70, 45]

Answer 68

names = ['United States', 'Australia', 'Japan', 'India', 'Russia', 'Morocco', 'Egypt']
dr =  [True, False, False, False, True, True, True]
cpc = [809, 731, 588, 18, 200, 70, 45]

my_dict = {‘country’: names, ‘drives_right’: dr, ‘cars_per_cap’: cpc}

cars = pd.DataFrame(my_dict)

Question 69

Extract a row from dataframe in index column as dataframe

1) by a label
2) by number

Answer 69

1) df.loc[[‘ru’]]

2) df.iloc[[2]]

Question 70

Extract multiple rows and columns by their column and row labels.

1a) specific columns and rows by label
1b) all rows and specific columns by label

1a) specific columns and rows by number
1b) all rows and specific columns by number

Answer 70

1a) df.loc[[‘ru’,’de’],[‘country’,’capital’]]
1b) df. loc[:,[‘country’,’capital’]]

2a) df.iloc[[1,2],[2,3]]
2b) df.iloc[:,[2.3]]

Question 71

given:
dataframe with index | country | cars_per_cap

output:
for each row print:
“country: cars_per_cap”

Answer 71

for lab, row in cars.iterrows() :

print(lab + ‘: ‘ + str(row[‘cars_per_cap’]))

Question 72

given:
dataframe with index | country

output:
append the dataframe with a new column COUNTRY that has country in upper case

Answer 72

cars[‘COUNTRY’] = cars[‘country’].apply(str.upper)