Importing Flat Files & Other Data

Question 1

how to access the system shell in IPython (on DataCamp)

Answer 1

!

Question 2

display directory contents

Answer 2

! ls

Question 3

open a text file as read-only

Answer 3

open(‘file.txt’, ‘r’)

Question 4

print an open file

Answer 4

print(file.read())

Question 5

check if a file is closed

Answer 5

file.closed

Question 6

close a file

Answer 6

file.close()

Question 7

alternative to opening and closing a file

Answer 7

context manager: with open(‘file.txt’) as file:

Question 8

read one line of a file

Answer 8

file.readline()

Question 9

flat files

Answer 9

table data without structural relationships (like a database would have)

Question 10

packages to import flat files

Answer 10

NumPy or pandas

Question 11

how to import a flat file with NumPy

Answer 11

np.loadtext(file, delimiter=, skiprows=, usecols=, dtype=

Question 12

tab delimiter

Answer 12

‘\t’

Question 13

how to import mixed datatypes with NumPy

Answer 13

np.genfromtxt(file, delimiter=, names=, dtype=None)

Question 14

names argument

Answer 14

if =True, tells us there is a header

Question 15

what does genfromtxt() produce

Answer 15

a structured array; 1D array where each element is a row of the flat file imported

Question 16

access row of a structured array

Answer 16

array[index]

Question 17

access column of a structured array

Answer 17

array[‘Column name’]

Question 18

similar to genfromtxt() with default argument dtype=None

Answer 18

np.recfromcsv()

Question 19

np.recfromcsv() defaults

Answer 19

delimiter=’,’ names=True dtype=None

Question 20

importing flat file with pandas as DataFrame

Answer 20

pd.read_csv(‘file’)

Question 21

converting a DataFrame to numpy array

Answer 21

df.values

Question 22

missing values in a DataFrame

Answer 22

NA or NaN (use na_values argument to specify string to replace)

Question 23

pandas equivalent of delimiter

Answer 23

sep=

Question 24

comment argument

Answer 24

removes comments after a given character (eg: comment=’#’)

Question 25

explore working directory in Python

Answer 25

import os

os.listdir(os.getcwd())

Question 26

importing pickle files

Answer 26

import pickle

pickle.load(file) (after first opening up the context manager)

Question 27

import Excel with pandas

Answer 27

pd.ExcelFile(file)

Question 28

Excel sheet names

Answer 28

spreadsheet.sheet_names

Question 29

import a given sheet

Answer 29

spreadsheet.parse(‘specific sheet’)

Question 30

how to import SAS files

Answer 30

import SAS7BDAT from sas7bdat

SAS7BDAT.to_data_frame(file)

Question 31

context manager for SAS files

Answer 31

with SAS7BDAT(‘file’) as file:

Question 32

import stata (.dta) files with pandas

Answer 32

pd.read_stata(‘file’)

Question 33

importing HDF5 files

Answer 33

h5py.File(file, ‘r’)

Question 34

importing MATLAB files

Answer 34

import scipy.io

scipy.io.loadmat(‘filename’)