Exploring data

Question 1

Visualize how to get the variance of a dataframe using groupby

Answer 1

df.groupby(by=”col1”)[[“col2”,”col3”,”col4”]].var()

Question 2

Visualize how to use .describe() on groups to get measurements by the percentiles parameter.

Answer 2

df.groupby(by=”col1”)[[“col2”,”col3”,”col4”]].describe(percentiles=[0.25,0.5,0.75])

Question 3

What is a Histogram? Visualize how to create.

Answer 3

df.plot(kind=”hist”)

It displays the distribution of numerical data
It divides data into bins and shows frequency of observations in each bin

Question 4

Pandas: Visualize how to create a bar chart

Answer 4

df.plot(kind=”bar”)

It compares different categories and shows values as bars of various lengths.

Question 5

matplotlib: Visualize how to create a pie chart

Answer 5

labels = “L1”, “L2”, “L3”
sizes = [10,20,25]

fig, ax = plt.subplots()
ax.pie(sizes, labels=labels, autopct=’%1.1f%%’, pctdistance=1.25, labeldistance=.6, colors=[“C1”,”C2”,C3”])

Use pctdistance and labeldistance if you want the percentages outside of the pct.

Question 6

A. Visualize how you case use aggregation on all columns
B. visualize how you can use .agg() on specific columns
C. Visualize how you can use .agg() using .groupby()
D. Visualize how to rename columns with .aagg

Answer 6

A. df.agg([‘mean’, ‘sum’, ‘max’])
B. df.agg({ ‘col1’: ‘mean’, ‘col2’: [‘sum’, ‘min’], ‘col3’ : lambda x: x.std()})
C. df.groupby(‘col_group’).agg({‘col1’: ‘mean’, ‘col2’: ‘sum’, ‘col3’ : ‘max’})
D. df.groupby(‘group_column’).agg(mean_col1=(‘col1’, ‘mean’), sum_col2=(‘col2’, ‘sum’)

Question 7

Visualize how to reset the index of a DataFrame

Answer 7

df.reset_index()

Question 8

Visualize an example of how to use groupby to calculate mean

Answer 8

data = { ‘model’: [‘Car A’, ‘Car A’, ‘Car B’, ‘Car B’, ‘Car C’], ‘city_mpg’: [20, 22, 25, 27, 18]}
df = pd.DataFrame(data)
mean_mpg = df.groupby(‘model’)[‘city_mpg’].mean()
print(mean_mpg)
Output -
model city_mpg
Car A 21.0,
Car B 26.0 ,
Car C 18.0

Question 9

A. Visualize how to calculate the mean on a dataframe.
B. Visualize how to calculate the mean on a column

Answer 9

A. df.mean(numeric_only=True)
B. df.groupby(“col”).mean(numeric_only=True)

Question 10

What does standard deviation measure?

Answer 10

How much each point differs from the mean, or how spread out the data is.

.std()

Question 11

What is variance?

Answer 11

Variance helps us understand how the numbers in a group differ from the average, giving a sense of how scattered or clustered the data is.
​
.var()

Question 12

What are quantiles? And how do you use them?

Answer 12

Quantiles are values that split a group of data into equal parts.

df[[‘col1’,’col2’,’col3’]].qunatile(q=[.25,.50,.75,1])

You can change the percentages to be whatever you want.

Question 13

What method would you use to show capital gains and capital loss?

Answer 13

dataframe[[“capital-gain”, “capital-loss”]].sum()

Question 14

What are the different panda plotting methods?

Answer 14

A. df.hist(figsize=(#,#)); or df[col].hist(figsize=(8,8));
B. df.plot(kind=”box”, figsize=(#,#)) or df[col].plot(kind=”box”, figsize=(#,#))
C. df.bar() or df[col].bar()
D. df.pie() or df[col].pie()
E. pd.plotting.scatter_matrix()
F. df.scatter() or df[col].scatter()
G. df.box() or df[col].box()

Question 15

How would you plot a bar chart with the value counts?

Answer 15

df[‘col’].value_counts().plot(kind=’bar’);

Question 16

Using the df.plot(kind=’scatter’), visualize how to specify what axis each column will be plotted.

Answer 16

df.plot(x=’col1’, y=’col2’, kind=’scatter’);

Question 17

A. Visualize how to use .hist() return a matplotlib subplot, change its transparency, and change the figure size
B. Visualize how to layer a new histogram using the same subplot that was returned.

Answer 17

A. ax = df_A[‘col1’].hist(alpha=0.5, figsize=(#,#,), label = ‘title1’);
B. df_B[‘col1’].hist(alpha=0.5, figsize=(#,#,), label=’title2’, ax=ax);
C.
ax.set_title(‘TITLE’);
ax.set_xlabel(‘X-AXIS TITLE’);
ax.set_ylabel(‘Y-AXIS TiTLE’);
ax.legend(loc=’upper right’);

Question 18

If you had a column containing the sales for each week for an entity, how would you find the row corresponding to the minimum sales or the worst week?

Answer 18

.idxmin()

ex:
#Step 1: find the index of the minimum sales
worst_week_index = df[‘col’].idxmin()
#Step 2: Access the corresponding week
worst_week = df.loc[worst_week_index, ‘week’]
#Print
print(f”The worst week is: {worst_week}”}

Question 19

A. Visualize how to customize the x-axis and the y-axis of your Pandas plots.
B. Visualize how to set the minimum and maximum values of your Pandas plot

Answer 19

A.
df.plot(xlabel=’X Axis Label’)
df.plot(ylabel=’Y Axis Label’)

B.
df.plot(ylim=(min_value,max_value))

Question 20

Visualize how to set the color for your Pandas plot

Answer 20

df.plot(color=’color_name’)

You can use color names or hexadecimal color codes.

Question 21

Visualize how to set hte legend for your Pandas plot

Answer 21

df.plot(legend=True)

Question 22

How can you use .index with .value_counts()?

Answer 22

You can use .index with .value_counts to access the unique values in a column along with their counts

EX.
# Example DataFrame
data = {‘fruits’: [‘apple’, ‘banana’, ‘apple’, ‘orange’, ‘banana’, ‘apple’]}
df = pd.DataFrame(data)
# Calculate value counts and index
counts_index = df[‘fruits’].value_counts().index
#Print
print(counts_index)
#Output
Index([‘apple’, ‘banana’, ‘orange’], dtype=’object’)

Question 23

What does the .index do?

Answer 23

The .index attribute gives you the unique values (the categories)

Question 24

What does .values do?

Answer 24

The .values attribute returns a NumPy array including duplicates, in order of appearance.
This is useful for nummerical computations or array manipulations or for converting to Python lists such as df[‘col’].values.tolist()

Question 25

What is the difference between a pandas series and a NumPy array?

Answer 25

Panda series are designed for labeled data which makes it easier to work with tabular data, and is built on top of NumPy arrays.

NumPy Arrays are designed for numerical computations and linear algebra operations. This has no labels or metadata.

Question 26

What do you do when you encounter a column that contains rows with lists?

Answer 26

Use the .explode() method to distribute the values across multiple rows in a column

df.explode(column=”column_name”)

Question 27

What do you need to do if you want to use .explode() and then distribute the values across multiple columns?

Answer 27

First flatten the list into individual rows in a column

df_explode = df.explode(column=”column_name”)
#Then split the data into multiple columns
df_new = pd.DataFrame(df_explode[“column_name”].tolist(), columns=[“col1”, “col2”, “col3”])

Question 28

What does .tolist() do?

Answer 28

This conversts a pandas Series or a DataFrame column into a python list

Question 29

Explain what pd.Dataframe({“list_values”:[str(a) for a in array.tolist()]}) does

Answer 29

• array.tolist() convers the NumPy array into a python list
• [str(a) for a in array.tolist()] irerates over each list a in the list and converst list a into its string representation.
• pd.DataFrame() converts the data into a column with the name list_values with each entry a corresponding string representation of the list from the original NumPy array

Question 30

Explain what pd.DataFrame(dataframe.column.apply(lambda u: eval(u)).values.tolist()) accomplishes on a column that contains string representations of lists

Answer 30

• dataframe.column is the column that we are accessing that contains the string list.
• .apply (lambda u: eval(u)) uses the apply method to apply the lambda function u: eval(u) to the dataframe.column
• .eval() converts the string representation of the list back into a python list.
• .values retries the underlying data as a NumPy array, which will now be a 1D array of lists
• .tolist() converts the NumPy array of lists into a Python list of lists
• pd.DataFrame creates a new DataFrame from the list of lists with each inner list becoming a row in the new DataFrame

Question 31

If I wanted to take.a column containing rows of string representations and convert it back into actual lists, what code would I use?

Answer 31

pd.DataFrame(datafame.column.apply(lambda u: eval(u)).values.tolist())

Question 32

What is