big data, functions, and more paper 2 stuff Flashcards

1
Q

a database is stored at an office. staff use a client server database system. describe an example of a problem that could occur if no systems were in place to manage concurrent access to the database

A
  • two users edit the same data simultaneously
  • one user writes the data then the other user writes the data
  • only one users update is lost
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q
  • the performance of a system is unsatisfactory. the time delay between a client sending a query to the server and the client receiving the results is long
  • explain how the performance of the system might be improved, include the following factors:
  • the hardware of the server
  • the design of the computer network
  • the database and software running on the server
A
  1. Server hardware
    - replace the processor with one that had more cores
    - replace processor with one that has more cache memory
    - use a processor with a bigger word size
    - faster clock speed
    - replace HDD’s with SSDs
    - use the harvard architecture
  2. Network
    - replace the network cable with a cable that has higher bandwidth
    - use a star topology instead of bus
    - consider using a more efficient protocol for the data across the network
    - add additional wireless access points
  3. Database and Software
    - use a more efficient technique for controlling concurrent access to the database
    - ensure the software is compiled rather than executed by an interpreter
    - use a non relational database
    - distribute the data across multiple servers
    - archive data tat is no longer necessary
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

explain some of the challenges that face legislators in the digital age

A
  • information can be combined/transferred in ways that were not previously possible
  • technology evolves quickly so it’s difficult for the law to keep up with changes
  • different laws in different countries
  • some crimes may be committed by states rather than individuals
  • methods such as encryption makes it harder to monitor criminal activity
  • individuals may have access to large amounts of sensitive information that may be of public interest
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

what is a programming paradigm

A

a style of computer programming

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

what is procedural programming

A
  • supported by languages like Python/Pascal
  • have a series of instructions that tell the computer what to do with the input in order to solve the problem
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

what is structured programming

A

a type of procedural programming which uses the programming constructs of sequence , selection, iteration and recursion
- uses modular techniques to split large programs into manageable chunks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what is oop

A
  • supported by languages like Java, Python, Delphi
  • type of programming that organises software design around data or objects
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what is declarative programming

A
  • used by sql
  • you write statements that describe the problem to be solved
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

what is functional programming

A
  • supported by Haskell, Python, Java, C++ and more
  • statements are written as a series of functions which accept data as arguments and return an output
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

what is a function

A
  • a mapping from a set of inputs
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

what is the domain

A

the set of input values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what is the codomain

A

set of output values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

the domain or codomain have to be…

A

part of a data type

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

isEqual :: int -> int -> bool

A

int = input time
isequal = function name
bool = what gets returned

isEqual :: x y = x == y
return True if x = y

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

what is a first class object

A

an object which may:
- appear in expressions
- be assigned to a variable
- be assigned as an argument
- be returned in a function call

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

differences between functional
and procedural

A
  • in functional, the value of a variable cannot change. program is said to be stateless
  • the only thing a function can do is calculate something and return a result, no side effects
17
Q

what is functional composition

A

when we combine two functions to get a new function

18
Q

partial function application

A
  • process + one of the variables
    eg (add 6) 4
19
Q

what are high order functions

A
  • any function that takes a function as an argument or returns a function as a result or does both
20
Q

what is map

A
  • a higher order function that takes a list and the function to be applied to the elements in the list
  • makes a list by applying the function to each element in old list

eg map (max 3) [1,2,3,4,5] = [3,3,3,4,5]

21
Q

what is filter

A
  • another high order function which takes a predicate(to define a Booleon condition) and a list
  • it returns the elements within a list that satisfies the Bool condition
    eg
    filter (==5) [2,5,7,2,5] = [5,5]
22
Q

what is the fold function

A
  • refuse a list to a single value using recursion, by applying an operator

eg to add all the elements in a list, function will be foldl (+) 0[2,3,4,5]

23
Q

foldl foldr

A

foldl = fold left
eg, foldl (/) 100[2,5] = (100/2) / 5 = 5

foldr = fold right
eg foldr (/) 100[2,5] = 2/(5/100) = 40

24
Q

heads and tails of a list

A
  • heads is the first item in the list
  • tails is the rest of the list
    eg [1,2,3,4]
  • 1 is the head
  • [2,3,4] is the tail
25
why is fold high order function
- the operations are functions
26
what is big data
data that is collected on such a large scale that it cannot easily be analysed
27
characteristics of big data
- velocity , the data is generated at high velocity - high VOLUME of data to process, cannot all be stored on one server - variety , big data comes in many forms, structured, unstructured, text, video, image
28
features of functional programming that make it easier to write
- statelessness, the programs behaviour doesnt depend on the order in which the functions are called - immutable data structures, which cannot be invertedly altered in a function - higheer order functions such as map/fold allow functions to be input as arguments
29
map and fold/reduce are efficiently parallelised, what does this mean
many processors can work simultaneously on parts of a dataset without affecting other parts
30
what do the circles represent in big data
entities
31
what do the squares represent in big data
fields associated with entities
32
what are facts based models
- models that captures a single piece of information, eg sensor location
33
how can a function be partially applied to an argument (add for example)
- the function is applied to one of the arguments - the output of this function is a new function - eg (add 4) 6
34
Describe what Big Data is, using examples
- data collected on a large scale characteristics: - Variety of different forms of information, eg :Email messages, Videos - high volume of data, eg large medical datasets for diagnosis - The data is must be processed ay high velocity, eg :Thousands of items to process per second.
35
challenges that could arise with big data
- Data cannot be stored on one server * Not possible to process data quickly enough with one computer. * Data cannot be represented in a table * Some forms of data / unstructured data are difficult to analyse.
36
how challenges with big data can be overcome
* Distributed database systems distributed across multiple servers. * Use of functional programming. * parallelising the execution of programs. * input split into parts then mapper executed on each part then all results combined by reducers * Functional programming makes it easier to write distributable code * Functional programming makes it easier to write correct code * Use of many thousands of commodity servers. * Use of servers with multiple CPUs / cores / drives. * Machine learning can identify patterns / the value in the data // use of predictive data models. * Use of languages such as XML or JSON to describe semi-structured data.
37
what do the circles in the graph represent in big data
the nodes holding the PROPERTIES that are atomic
38
what do the edges in big data graphs represent
the interactions/relationships between the nodes