Foundations - Roles and Concepts

Question 1

what are the issues organization will have

Answer 1

Data Processing in SILOS
Excessive Data movement
Data Duplication

Question 2

Data Engineer role

Answer 2

Raw data into valuable insights
Design, Develop and Maintain data architectures ad (ETL)

Getting Data from sources, Making it useful and Serving to stakeholders

Question 3

What are the Key functions of DE

Answer 3

Build and Maintain Data Infrastructure
Ingest data from Various sources
Prepare ingested data for analytics
Catalog and document curated datasets
Automate regular data flows
Ensure Data Quality, Security and Compliance

Question 4

Build and Maintain Data Infrastructure

Answer 4

Setting up databases, data lakes, and data warehouses on AWS services like Amazon Simple Storage Service (Amazon S3), AWS Glue, Amazon Redshift, among others

Question 5

Ingest data from Various sources

Answer 5

Use tools like AWS Glue jobs or AWS Lambda functions to ingest data from databases, applications, files, and streaming devices into the centralized data platforms.

Question 6

Prepare ingested data for analytics

Answer 6

Use technologies like AWS Glue, Apache Spark, or Amazon EMR to prepare data by cleaning, transforming, and enriching it.

Question 7

Catalog and document curated datasets

Answer 7

Use AWS Glue crawlers to determine the format and schema, group data into tables, and write metadata to the AWS Glue Data Catalog. Use metadata tagging in Data Catalog for data governance and discoverability

Question 8

Automate regular data flows and Pipelines

Answer 8

Simplify and accelerate data processing using services like AWS Glue workflows, AWS Lambda, or AWS Step Functions.

Question 9

Ensure Data Quality, Security and Compliance

Answer 9

Create access controls, establish authorization policies, and build monitoring processes. Use Amazon DataZone or AWS Lake Formation to manage and govern access to data using fine-grained controls. These controls help ensure access with the right level of privileges and context

Question 10

Chief data officer CDO - Responsibility

Answer 10

Builds a culture of using data to solve problems and accelerate innovation

Question 11

Chief Data Office - Area of Interest

Answer 11

Data quality, data governance, data and artificial intelligence (AI) strategy, evangelizing the value of data to the business

Question 12

Data architect - Responsibility

Answer 12

Driven to architect technical solutions to meet business needs, focuses on solving complex data challenges to help the CDO deliver on their vision

Question 13

Data Architect - Area of Interest

Answer 13

Data pipeline, data processing, data integration, data governance, and data catalogs

Question 14

Data engineer - Responsibility

Answer 14

Delivers usable, accurate datasets to the organization in a secure and high-performing manner

Question 15

Data Engineer - Area of Interest

Answer 15

The variety of tools used for building data pipelines, ease of use, configuration, and maintenance

Question 16

Data security officer - Responsibility

Answer 16

Ensures that data security, privacy, and governance are strictly defined and adhered to

Question 17

Data Security Officer - Area of Interest

Answer 17

Keeping information secure, complying with data privacy regulations, protecting personally identifiable information (PII), and applying fine-grained access controls and data masking

Question 18

Data scientist - Responsibility

Answer 18

Constructs the means for quickly extracting business-focused insight from data for the business to make better decisions

Question 19

Data Scientist - Area of Interest

Answer 19

Tools that simplify data manipulation and provide deeper insight than visualization tools and tools that help build the machine learning (ML) pipeline

Question 20

Data analyst - Responsibility

Answer 20

Reacts to market conditions in real time, must have the ability to find data and perform analytics quickly and easily

Question 21

Data Analyst - Area of Interest

Answer 21

Querying data and performing analysis to create new business insights and producing reports and visualizations that explain the business insights