Module One - Data Science Tools

Question 1

Name the Data categories of Data Science tasks

Answer 1

Data Management
Data Integration and Modification
Data Visualization

Question 2

Name the Model categories of Data Science tasks

Answer 2

Model Building
Model Deployment
Model Monitoring and Assessment

Question 3

How are Data Science tasks supported?

Answer 3

Data Science Tasks are supported by:
Code Asset Management
Data Asset Management
Execution Environments
Development Environments

Question 4

What is the main characteristic of Jupyter Notebooks?

Answer 4

A key property of Jupyter Notebooks is to unify documentation, code, output from the code, shell commands, and visualizations in a single document.

Question 5

Suggest a tool suitable for web applications that handle large volumes of unstructured data

Answer 5

mongoDB is a document-oriented NoSQL database that stores data in flexible JSON (Binary JSON).

Question 6

IBM tools that cover the complete Data Science, ML and AI life cycle

Answer 6

Watson Studio and Watson OpenScale, cover the complete development life cycle for all data science, machine learning, and AI tasks.

Question 7

What is RStudio?

Answer 7

Studio is an integrated development environment (IDE) designed specifically for working with the R programming language, which is a popular tool for statistical computing and data analysis.

Question 8

What is the de facto standard tool in Code Asset Management?

Answer 8

Git is an quintessential code asset management tool. GitHub is a popular web-based platform for storing and managing source code. Its features make it an ideal tool for collaborative software development, including version control, issue tracking, and project management.

Question 9

Main difference between Apache Spark and Apache Hadoop for Big Data

Answer 9

Apache Hadoop - distributed file system for large datasets
Apache Spark - analytics engine for large-scale data processing, with support for batch, streaming, and ML workloads

Question 10

Is Apache Cassandra suitable to allow your database services to scale across many commodities servers?

Answer 10

Yes. Apache Cassandra is a highly scalable, distributed NoSQL database that handles large amounts of structured and unstructured data across servers.

Question 11

Describe Apache Airflow

Answer 11

Apache Airflow is an open source platform for programmatically authoring, scheduling and monitoring workflows.

Question 12

Is Kubernetes a proper tool for scaling, managing and launching containerized applications?

Answer 12

Yes. Kubernetes is an open source platform for container orchestration. It automatically launches, scales and manages containerized applications.

Question 13

Which Apache tool is suitable for analyzing and visualizing large data sets in Apache Hadoop?

Answer 13

Hue is an open source web interface for analyzing and visualizing large data sets in Apache Hadoop

Question 14

What is the other name used for Data Asset Management?

Answer 14

Data asset management is also known as data governance or data lineage.

Question 15

What is the key advantages of APIs in model deployment?

Answer 15

Real-time predictions.
Enable real-time interaction between models and applications. APIs are the most common enabler of real-time, data-based decisions as they allow external systems to make predictions on the fly, often combined with CI/CD automation (e.g., GitLab) for seamless deployment and updates.

Question 16

What is the key advantage of Packages during model deployment?

Answer 16

Packaging models as software libraries or containers can help with reproducibility, version control, and easier deployment.

Question 17

How does APIs work during the model deployment stage?

Answer 17

A model is deployed as a service via an API (often a REST or gRPC API), allowing external systems to send data to the model and receive predictions in return. This setup facilitates seamless integration into web applications, mobile apps, and other software systems.

Question 18

How does Packages work during the model deployment stage?

Answer 18

A model is wrapped into a package (e.g., Python .whl or .tar.gz files) or a container (e.g., Docker), making it easy to deploy across environments or integrate into other software systems. Packaging can also be done using tools like TensorFlow Serving or PyTorch Serve to serve models at scale.

Question 19

Describe the key features of DataMeer data analytics platform

Answer 19

1. Data Integration
2. Self-Service Data Preparation (Users can clean, join, filter, and transform datasets using a visual interface)
3. Advanced Complex Analytics
4. Data Exploration and Visualization
5. Collaboration and Governance

Question 20

What is the key purpose of the DataMeer data analytics platform?

Answer 20

DataMeer is designed to simplify and speed up the process of preparing, analyzing, and visualizing large datasets.
It is primarily used for data preparation, business intelligence (BI), and data exploration.

Question 21

Describe the key features of DataRefinery data transformation platform

Answer 21

Data Preparation and Transformation
Scalability
Automation
Collaboration
Data Governance and Compliance

Question 22

What is the key purpose of the DataRefinery data transformation platform?

Answer 22

DataRefinery is designed to simplify and automate the process of preparing, cleaning, and transforming large datasets from various sources for analytics and machine learning with focus on scalability. It helps data teams streamline the often time-consuming and complex tasks of data wrangling, enabling faster and more efficient analysis.

Question 23

Is DataRefinery platform tool a part of IBM Watson Studio?

Answer 23

Yes. Data Refinery is part of IBM Watson Studio.

Question 24

What is the main advantage of Apache Airflow platform?

Answer 24

Apache Airflow excels at automating, orchestrating, and monitoring workflows in data-intensive environments. It’s highly versatile, allowing it to be used for a wide range of tasks from data engineering and ETL, to machine learning, to infrastructure management, and more.

Question 25

Give an example of an use case of ETL with Apache Airflow platform

Answer 25

Automating Extract Transform and Load data pipelines.
Extracting sales data from multiple APIs or databases, transforming it to unify formats, and loading the processed data into a cloud data warehouse like Amazon Redshift or Google BigQuery.

Question 26

Give an example of an use case of ETL with Apache Airflow platform

Answer 26

Airflow is often used to manage and automate the movement of data into and within data warehouses.
Example: Automating the nightly data load into a Snowflake warehouse, where new data is appended, and old data is archived or updated.

Question 27

Give an example of an use case of implementing ML pipelines with Apache Airflow platform

Answer 27

Airflow helps manage machine learning workflows by scheduling model training, validation, hyperparameter tuning, and model deployment. It ensures that the steps in the pipeline are run in a specific order with dependencies.
Example: Running daily batch jobs to retrain a machine learning model with new data, then evaluating the model’s performance and deploying it if it meets certain accuracy thresholds.

Question 28

What are the benefits of using Fiddler for model monitoring in Data Science?

Answer 28

Fiddler is essential in data science for ensuring that machine learning models in production continue to perform well, remain interpretable, avoid bias, and comply with industry regulations, enabling data teams to maintain transparency and trust in AI systems.

Proactive Management: Fiddler helps teams catch problems in production early.

Accountability and Trust - By providing explainability, Fiddler builds trust in machine learning models

Ethical AI Development: It helps ensure that models are free from bias and are ethically sound

Scalability: Fiddler scales across multiple models

Question 29

Describe the key features of Apache Airflow

Answer 29

1. Workflow as Code: define workflows as Directed Acyclic Graphs (DAGs) using Python code.
2. Task Scheduling, Parallel Execution and Task Orchestration
3. Extensibility and Integration with AWS, Google Cloud Platform, Apache Spark, Hadoop, Databricks, databases, and many more.
4. Horizontal Scalability
5. Web-based UI for Monitoring and Visualization: