Importing Flat Files & Other Data

Question 1

how to access the system shell in IPython (on DataCamp)

Answer 1

!

Question 2

display directory contents

Answer 2

! ls

Question 3

open a text file as read-only

Answer 3

open(‘file.txt’, ‘r’)

Question 4

print an open file

Answer 4

print(file.read())

Question 5

check if a file is closed

Answer 5

file.closed

Question 6

close a file

Answer 6

file.close()

Question 7

alternative to opening and closing a file

Answer 7

context manager: with open(‘file.txt’) as file:

Question 8

read one line of a file

Answer 8

file.readline()

Question 9

flat files

Answer 9

table data without structural relationships (like a database would have)

Question 10

packages to import flat files

Answer 10

NumPy or pandas

Question 11

how to import a flat file with NumPy

Answer 11

np.loadtext(file, delimiter=, skiprows=, usecols=, dtype=

Question 12

tab delimiter

Answer 12

‘\t’

Question 13

how to import mixed datatypes with NumPy

Answer 13

np.genfromtxt(file, delimiter=, names=, dtype=None)

Question 14

names argument

Answer 14

if =True, tells us there is a header

Question 15

what does genfromtxt() produce

Answer 15

a structured array; 1D array where each element is a row of the flat file imported

Question 16

access row of a structured array

Answer 16

array[index]

Question 17

access column of a structured array

Answer 17

array[‘Column name’]

Question 18

similar to genfromtxt() with default argument dtype=None

Answer 18

np.recfromcsv()

Question 19

np.recfromcsv() defaults

Answer 19

delimiter=’,’ names=True dtype=None

Question 20

importing flat file with pandas as DataFrame

Answer 20

pd.read_csv(‘file’)

Question 21

converting a DataFrame to numpy array

Answer 21

df.values

Question 22

missing values in a DataFrame

Answer 22

NA or NaN (use na_values argument to specify string to replace)

Question 23

pandas equivalent of delimiter

Answer 23

sep=

Question 24

comment argument

Answer 24

removes comments after a given character (eg: comment=’#’)

Question 25

explore working directory in Python

Answer 25

import os | os.listdir(os.getcwd())

Question 26

importing pickle files

Answer 26

import pickle | pickle.load(file) (after first opening up the context manager)

Question 27

import Excel with pandas

Answer 27

pd.ExcelFile(file)

Question 28

Excel sheet names

Answer 28

spreadsheet.sheet_names

Question 29

import a given sheet

Answer 29

spreadsheet.parse('specific sheet')

Question 30

how to import SAS files

Answer 30

import SAS7BDAT from sas7bdat | SAS7BDAT.to_data_frame(file)

Question 31

context manager for SAS files

Answer 31

with SAS7BDAT('file') as file:

Question 32

import stata (.dta) files with pandas

Answer 32

pd.read_stata('file')

Question 33

importing HDF5 files

Answer 33

h5py.File(file, 'r')

Question 34

importing MATLAB files

Answer 34

import scipy.io | scipy.io.loadmat('filename')