Pig Scripts

Question 1

How to filter

Answer 1

FILTER dataset BY expression

Question 2

generate min/max value of column

Answer 2

FOREACH (GROUP dataset ALL) GENERATE MAX(dataset.column) AS var_name;

Question 3

Calculate difference between min max

Answer 3

max_project = FOREACH (GROUP grades ALL) GENERATE MAX(grades.project) AS max_project;

– Calculate the worst ‘project’ value
min_project_high_exam = FOREACH (GROUP high_exam_students ALL) GENERATE MIN(high_exam_students.project) AS min_project_high_exam;

– Calculate the difference
difference = FOREACH max_project GENERATE max_project - min_project_high_exam AS project_difference;

Question 4

Storing results for further processing

Answer 4

STORE var_name INTO table USING PigStorage(‘,’);

Question 5

Load dataset with schema

Answer 5

X = LOAD ‘table’ USING PigStorage(‘,’) AS (ID:int, ca1:int);

Question 6

Key functions {average, concatenate, count, difference, max/min, sum}

Answer 6

AVG
CONCAT
COUNT
DIFF
MAX/MIN
SUM

Question 7

For Loop

Answer 7

FOREACH X GENERATE var_type{tuple, bag}

Question 8

How to group

Answer 8

GROUP dataset BY dataset.column;

Question 9

When to define the schema

Answer 9

When loading the dataset

Question 10

How to sort

Answer 10

ORDER dataset BY dataset.column