Pandas

Question 1

What is Pandas?

Answer 1

It is a newer package built on top of NumPy, providing an efficient implementation of a DataFrame. These are multi-dimensional arrays with attached row and column labels, often allowing heterogeneous types.

Question 2

What are advantages of Pandas over NumPy?

Answer 2

Pandas provides efficient access to these sorts of “data munging” tasks that occupy much of a data scientist’s time and for which NumPy is not sufficient.

NumPy limitations become clear when we need more flexibility (e.g., attaching labels to data, working with missing data, etc.) and when attempting operations that do not map well to element-wise broadcasting (e.g., groupings, pivots, etc.), each of which is an important piece of analyzing the less structured data available in many forms in the world around us

Question 3

What are the three fundamental Pandas data structures?

Answer 3

Series, DataFrame and Index.

Question 4

What is a Pandas Series?

Answer 4

A Pandas Series is a one-dimensional array of indexed data values. The Series wraps a sequence of values and indices.

Question 5

How do you create a Pandas Series?

Answer 5

It can be created from a list or array as follows:

From list:
data = pd.Series([0.25, 0.5, 0.75, 1.0])

From array
x = np.arange(4,10)
data = pd.Series(x)

Question 6

What does the values attribute of a Series return?

Answer 6

A NumPy array of the data values

Question 7

What does the index attribute of a Series return?

Answer 7

An array-like object of type pd.Index containing the index of the Series.

Question 8

How can you access elements of a Series?

Answer 8

Use the associated index e.g. for a Series x

x[0] # returns first element
x[0:3] # returns first 3 elements along with their index

Question 9

What is the essential difference between a Series and a NumPy array?

Answer 9

The presence of the index: while the Numpy Array has an implicitly defined integer index used to access the values, the Pandas Series has an explicitly defined index associated with the values.

Question 10

Does a Series index need to be an integer?

Answer 10

No, it can be values of any type e.g.

data = pd.Series([0.25, 0.5, 0.75, 1.0],
index=[‘a’, ‘b’, ‘c’, ‘d’])

Question 11

Can a Series be thought of as a specialization of a Python dictionary?

Answer 11

Yes. A Series is a structure that maps typed keys to a set of typed values. This makes it more efficient than Dictionaries for certain operations.

Question 12

What is a Pandas DataFrame?

Answer 12

A Pandas DataFrame is a two-dimensional array with flexible row indices and flexible column names.

Can think of DF as a sequence of aligned Series objects, where they share the same index.

Question 13

What does the index attribute of a DataFrame return?

Answer 13

An Index object containing the index of the DF.

Question 14

What does the columns attribute of a DataFrame return?

Answer 14

An Index object containing the column labels of the DF.

Question 15

How can a Pandas DataFrame be created from a Series?

Answer 15

pd.DataFrame(population, columns=[‘population’])

Where population is a Series

Question 16

How can a Pandas DataFrame be created from a Dictionary?

Answer 16

data = [{‘a’: i, ‘b’: 2 * i}
for i in range(3)]

pd.DataFrame(data)

NB. Even if some keys in Dict are missing, Pandas will fill in with NaN

Question 17

How can a Pandas DataFrame be created from a Dictionary of Series objects?

Answer 17

pd.DataFrame({‘population’: population,
‘area’: area})

Where population and area are Series.

Question 18

How can a Pandas DataFrame be created from a 2-D NumPy array?

Answer 18

pd.DataFrame(np.random.rand(3, 2),
columns=[‘foo’, ‘bar’],
index=[‘a’, ‘b’, ‘c’])

If columns and index value omitted, integer indexes will be used for both

Question 19

How is a Pandas Series like a Dictionary?

Answer 19

The Series object provides a mapping from a collection of keys to a collection of values (where index is the key)

data = pd.Series([0.25, 0.5, 0.75, 1.0],
index=[‘a’, ‘b’, ‘c’, ‘d’])

data[‘b’] # returns 0.5

Question 20

In what way can you extend a Series?

Answer 20

data[‘e’] = 1.25

where ‘e’ is the index
1.25 is the value

Question 21

Why is there potential confusion with slicing of Series objects?

Answer 21

Confusion can arise when slicing Series objects when they have explicit integer indexes. With a Series object called data:

Accessing data[1] will return the element at the explicit index of 1
Accessing data [1:3] will use the implicit python style and return values between the 2nd and 3rd elements.

Question 22

What is the preferred method of accessing Series for indexing and slicing?

Answer 22

The .loc and iloc attributes

data. loc[1] # the explicit index value of 1
data. loc[1:3] # the explicit slice from index 1 -> 3

data. iloc[1] # the implicit python index value at element 1
data. iloc[1:3] # the implicit python slice from 2nd -> 3rd elements

Question 23

How do we access individual column (Series) of a DataFrame?

Answer 23

Use dictionary style indexing with column name e.g.

data = pd.DataFrame({‘area’:area, ‘pop’:pop})
data[‘area’]

NB. Can also use data.area

Question 24

What does the values attribute of a DataFrame return?

Answer 24

A 2-Dimensional array of the data

Question 25

How do we use .iloc to access DataFrames?

Answer 25

Using the iloc indexer, we can index the underlying array as if it is a simple NumPy array (using the implicit Python-style index), but the DataFrame index and column labels are maintained in the result:

data.iloc[:3, :2]

Question 26

How do we use .loc to access DataFrames?

Answer 26

Using the loc indexer, we can index the underlying array as if it is a simple NumPy array (using the explicit index and column names), but the DataFrame index and column labels are maintained in the result:

data.loc[:’Illinois’, :’pop’]

Question 27

How do we use .ix to access DataFrames?

Answer 27

The .ix indexer is a hybrid of .loc and .iloc

data.ix[:3, :’pop’]

Question 28

What does data.loc[data.density > 100, [‘pop’, ‘density’]] demonstrate?

Answer 28

This is the use of masking (data.density >100) and fancy indexing ( [‘pop’, ‘density’] ) to select back data values from a DataFrame.

Question 29

Are Pandas object indices preserved when using NumPy ufuncs?

Answer 29

Yes

Question 30

What does NaN mean?

Answer 30

NaN - Not a Number, indicates missing data.

Missing values are filled in with NaN by default.

Question 31

What happens when concatenating DataFrames or Series objects with different indices/column headings?

Answer 31

The Pandas objects are effectively unioned, and any matching index/column values will be brought together. Elements which don;t match will be populated with NaN.

Question 32

When adding Series/DataFrames with non-matching index/columns what keyword can be used to replace NaN values?

Answer 32

Use the fill_value keyword to stipulate a filling method.

Question 33

Is the alignment of Series/DataFrames indices and columns preserved when performing operations in Pandas?

Answer 33

Yes. Broadcasting is used to ensure that the correct operations are applied to rows and columns.