U1T3.2 - Applications of DT

Question 1

What is data mining?

Answer 1

Process of analysis large data sets (big data) with view to discovering patterns + trends that go beyond simple analysis. Combines AI, stats + database systems in analysis of groups of (un)structured data sets which are difficult to analyse using traditional methods. Extracts info from data set + transforms into appropriate format for use. (Summary of input data for analysis) Stops at process of pattern extraction.

Question 2

What is big data?

Answer 2

Data sets so complex that traditional databases + other processing applications can’t capture, curate, manage + process them in acceptable time frame.

Question 3

What does curate mean?

Answer 3

Process of organising data from range of data sources.

Question 4

What are the 3 big data challenges?

Answer 4

Volume (amount of data to be processed), variety (num of types of data to be analysed) + velocity (speed of data processing)

Question 5

How does DT allow us to collect data for analysis?

Answer 5

Online forms, mobile phone data transmissions, email data, stock market data, market research, PDAs, smartphones, tablets + netbooks etc.

Question 6

How can data sources be categorised and what are the differences?

Answer 6

Internal + external. Internal = customer details, product details, sales data. External = business partners, data suppliers, internet, govt + market research companies.

Question 7

What are the most commonly used data sources?

Answer 7

Social media, machine data (generated from devices like RFID chip readers, GPS results) + transactional data (data from companies like eBay, Amazon, Tesco)

Question 8

What are the key requirements of big data storage?

Answer 8

Handle large amounts of data + keep scaling up to handle growth of data sets. High speed input/output operations to support delivery of data analytics as they’re carried out. Big data practitioners run hyperscale computing environments.

Question 9

What are hyperscale computing environments?

Answer 9

Consists of many servers with DAS, each unit has PCIe flash storage devices to support data storage + high speed access to data sets.

Question 10

What is DAS?

Answer 10

Direct Attached Storage.

Question 11

How can smaller organisations support the storage of big data?

Answer 11

Use of NAS devices, can scale outward so can be difficult to manage as span out in hierarchial manner (many devices, many folders within folders)

Question 12

What is NAS?

Answer 12

Network Attached Storage. File access shared storage, easily scaled out to meet increased capacity/computing requirements for big data analysis.

Question 13

What are object-based storage systems?

Answer 13

Alt to NAS devices + their issues. Each file storage given unique identifier + index to support high speed access to particular data file/set.

Question 14

What do big data processing techniques do?

Answer 14

Analyse data sets at terabyte/petabyte scale. Some methods include cluster analysis, classification, anomaly detection, association rule mining + sequential pattern mining, regression + summarisation.

Question 15

What is cluster analysis?

Answer 15

Groups of data records identified.

Question 16

What is classification?

Answer 16

Data mining process used to determine appropriate structure to new data. e.g. way email application classified some emails as spam.

Question 17

What is anomaly detection?

Answer 17

Unusual records identified. Some anomalies merit investigation as points of interest to organisation or may be representative of errors.

Question 18

What is association rule mining + sequential pattern mining?

Answer 18

Dependencies between data items identified. e.g. use of data sets by supermarket to determine which patterns of products bought together.

Question 19

What is regression?

Answer 19

Relationships between data variables investigated to help see how change in independent variable impacts on dependent data variable.

Question 20

What is summarisation?

Answer 20

Data summarised in visual format.

Question 21

What are some of the key objectives of collecting and using big data by the financial services sector?

Answer 21

Ensure they comply with regulations (using fuzzy matching to check customer names + aliases against customer blacklist, lower cost), improve risk analysis (algorithms run of transaction data to identify fraudulent activity/perform risk analysis, support trading decisions), understand customer behaviour/transaction patterns + improve services (identify what leads to dissatisfaction)

Question 22

How does the health sector use big data?

Answer 22

Predict epidemics, cure disease, improve life quality + avoid preventable deaths. Smartphones measure steps, diet + sleep patterns which in future could be shared with GP for diagnosis help. Supports clinical trials to select best subjects. Phone location can track pop movement and predict spread of Ebola virus.

Question 23

How does the retail sector use big data?

Answer 23

Predict trends + forecast demand, price optimisation (spending habits + demand) + identify potential customers (data collected through transactional records + loyalty programs allows demand to be forecast on basis of geographical areas)

Question 24

What is cloud computing?

Answer 24

Use of internet by large computing companies to provide services normally provided by LAN. Use server farms to host services they provide for other organisations who can access these services from any computer w/ internet connection. Users don’t know where data stored. Virtual servers form foundation of cloud servers. Capitalises on principle of virtual clusters.

Question 25

What are server farms?

Answer 25

Central computer centre consisting of large num of linked file servers. Each location could have many servers, comp storage devices + other components used to support services provided by Cloud Service Provider.

Question 26

What is virtualisation?

Answer 26

Allows virtual servers to run on physical server platform. Separates physical infrastructures to create dedicated resources. Possible to run multiple OS’s + applications on same server at same time by making servers. Manipulates hardware used to provide cloud computing as service to client users. Virtual version of physical device/resource where users can use resource as if real single resource.

Question 27

What is a cloud instance?

Answer 27

Location of physical memory on cloud server which has been allocated to particular client. Acts as virtual server for client + has own allocation of processing power, storage + other components. Each server has multiple clients. Can be used by client for processing of cloud based task/application. End user doesn’t need to consider how many servers/resources applied to application. Location is immaterial + dynamic nature means resources reassigned as needed without downtime. If end user wants access to app/service, create cloud instance for time they use app.

Question 28

What does each server in cloud computing having multiple clients do?

Answer 28

Possible to allocate additional capacity to clients when usage spikes + resource demands increase. Each of elements in instance is dynamic + can be changed

Question 29

What is ahosted solution?

Answer 29

Like cloud instance but hardware + software made available to client is reserved for servicing of their needs + noone else. They pay for all resources whether used or not. Where usage exceeds capacity, additional investment in resources by client is made.

Question 30

What are virtual clusters?

Answer 30

Formed when virtual machine, established to meet demands of cloud instance, configures available resources on network to meet client demands.

Question 31

How can users access cloud computing?

Answer 31

Secure client login to access services. Data transfer between cloud service providers + clients is via encrypted connection.

Question 32

What are some of the services available to clients with cloud computing?

Answer 32

Data storage, email, virtualised software, backup + remotely hosted applications.

Question 33

How does cloud computing provide data storage? What is cloud storage?

Answer 33

Saving data on off-site storage system provided by third party. Saved to remote database + connected to via internet.

Question 34

How does cloud computing provide email?

Answer 34

Web based services like Yahoo + Outlook allows users to access email from cloud using any browser/hardware platform with internet connection. Emails sent + received via client’s account stored on service provider’s server rather than being stored on user’s own comp.

Question 35

How does cloud computing provide virtualised software?

Answer 35

Many applications available via web. MS Office moved partly to web. Users can access application hosting + not consider storage space taken up on own personal devices. Software accessible without installing + updates automatic. Access from any location + can share + access data files from anywhere they have an internet connection.

Question 36

How does cloud computing provide backup?

Answer 36

Allows clients to store data on internet using storage service provider instead of locally on hard drive. Backups schedules automatically. To access backup, must use service provider’s specific application of web based interface provided.

Question 37

How does cloud computing provide remotely hosted applications?

Answer 37

Clients can access business applications from anywhere with internet + comp. Relevant software + data stored on remote server. Supports multiple concurrent access to apps + data by staff.

Question 38

What are the benefits of cloud computing?

Answer 38

Providers can react more responsibly to user requirement changes, collaboration from diff locations, updates readily available to all, increased security + reliability as only maintain 1 system, all users access 1 system so no cross-compatibility issues, reduced costs as don’t need to buy hard + software for meeting business needs, increased flexibility in work arrangements so changes in demand for computing resources can be provided for at a cost + increased reliability as data backup, recovery + business continuity easier + less expensive.

Question 39

What are the drawbacks of cloud computing?

Answer 39

Trust someone else to look after data (updates, backup, restoration, security), cyber attacks problematic, insider threats where employees with access to cloud data go undetected + could destroy entire cloud environment hosting data, users concerned over loss of data confidentially which can be accessed by govt + legal liability issues as become liable after data breach. (GDPR)

Question 40

What is server virtualisation?

Answer 40

Partitioning of physical server into smaller virtual servers.

Question 41

What is storage virtualisation?

Answer 41

Amalgamation of multiple network storage devices into single storage unit.

Question 42

What is operating system virtualisation?

Answer 42

Multiple operating systems used on single server.

Question 43

What is network virtualisation?

Answer 43

All servers + services in network treated as single pool of resources which can be accessed without regard for physical components.

Question 44

What is application virtualisation?

Answer 44

Each application adapts a set of configurations on demand.

Question 45

What are the 4 V’s of big data?

Answer 45

Volume (scale of data), velocity (analysis of streaming data), variety (diff forms of data) + veracity (uncertainty of data)