pandas2

Question 1

basic info. Give two methods. One of them also list categorical data/unique values/top counts

Answer 1

df.describe(include = ‘all’) and df.info()

Question 2

sort by values first in col1 then in col2, col1 ascending and col2 descending.

Answer 2

df = df.sort_values(by = [‘col1’, ‘col2’], ascending = [1, 0]).reset_index(drop = True)

Question 3

create dummy variable manually

Answer 3

df[‘col’] = df[‘col’].map({‘value1’: 1, ‘value2’: 0})

Question 4

x is a matrix of a list. you want to use elements in the list

Answer 4

x.columns.values

Question 5

delete columns

Answer 5

df.drop(columns = [‘col1’, ‘col2’], axis = 1)

Question 6

the value of the 99th percentile (to use later to remove outliers)

Answer 6

threshold = df[‘col’].quantile(0.99)

Question 7

create dummy variables for all

Answer 7

new_df = pd.get_dummies(df, drop_first=True)

Question 8

rearrange columns of a df, and what order should they be?

Answer 8

df.columns.values
## copy output array
cols = [PASTE HERE] # and rearrange
new_df = df[col]

dependent variable, numeric independent variable, dummies

Question 9

create a df after linear regression to check test results

Answer 9

y_test = y_test.reset_index(drop=True)
df = pd.DataFrame(np.exp(y_hat_test), columns=['Prediction'])
df['Target'] = np.exp(y_test)
df['Residual'] = df['Target'] - df['Prediction']
df_pf['Difference_perc'] = np.absolute(df_pf['Residual']/df_pf['Target']*100)

Question 10

set number of rows to display to n

Answer 10

pd.options.display.max_rows = n

Question 11

display floats to 2 digits

Answer 11

pd.set_option(‘display.float_format’, lambda x: ‘%.2f’ % x)

Question 12

check for nulls

Answer 12

df.isnull().sum()

Question 13

turn confusion matrix from statsmodels to pd df. Assume model is results_log

Answer 13

cm_df = pd.DataFrame(results_log.pred_table())
cm_df.columns = ['Predicted 0','Predicted 1']
cm_df = cm_df.rename(index={0: 'Actual 0',1:'Actual 1'})

Question 14

select all rows and first and second columns

Answer 14

df.iloc[:,1:3]