ML, Cloud computing and Edge computing Flashcards
Data mining (DM)
Focus on discovering patterns and knowledge from historical data to find valuable insights.
Machine learning (ML)
The ability of an algorithm to learn from data without being explicitly programmed for specific tasks.
Artificial Intelligence (AI)
Technologies and systems that act and think in a way similar to human behaviour, with the ability to make rational decisions.
Different types of machine learning algorithms(/types?)
Supervised machine learning, unsupervised machine learning and reinforcement learning.
Supervised machine learning
Classification: is when the output variable is a category, such as red or blue or disease and no disease.
Regression: a problem occurs when the output variable is a real value such as weight or price, e.g., house selling, stocks.
Algorithms for classification and regression in supervised machine learning?
Logistic regression: used to predict a binary outcome: yes/no, pass/fail.
Naive Bauers: calculate the possibility of whether a sample X belongs within a certain category or does not.
K-nearest neighbour: utilising training data to find the K closest relatives in future examples.
Decision tree: builds tree branches in a hierarchy approach, and each branch can be considered as an if-else statement.
Linear regression: a technique to examine whether there is a statistical relationship between a response variable and two or more explanatory variables (X).
Unsupervised machine learning
Splits datasets based on common attributes, detects anomalies that do not fit in any group and simplifies data by reducing dimensions. Clustering, anomaly detection and dimensionality reduction are common techniques. E.g., IoT in agriculture to segment plants based on sensor information.
Reinforcement learning
the machine is given feedback concerning the decision it makes, but no information about the possible alternatives. E.g., used in automated cars, and different games.
Challenges about ML in general
- Insufficient data: ML algorithms need a lot of data to work properly, simple problems may take thousands of examples and complex problems (image/speech recognition) may require millions.
- Poor quality data: if the training data is full of errors, outliners, and noises it will be hard for the system to detect underlying patterns.
- Irrelevant features: a critical part is coming up with a good set of features to train on.
- Feature selection: select the most useful features to train among existing features.
- Feature extraction: combining existing features to produce more useful ones, dimensionality reduction algorithms can help.
- Feature engineering: new features by gathering new data.
What is the role of ML in iot?
Predictive maintenance: finding signs of issues before a breakdown happens.
Anomaly detection: involves identifying events of data points that are outside the expected range.
Personalization: based on user behaviour and preferences, ml can be used to customize iot apps.
Environmental monitoring: data from sensors estimate environmental factors.
Resource optimization: ml can used to maximize the usage of resources like water, electricity and materials.
Smart transportation: cars, aeroplanes.
Challenges of using ML in IoT
What are some challenges in using ML in IoT?
Data quality: ML algorithms require high-quality data to provide accurate predictions.
Scalability: iot applications involve large amounts of data and a large number of devices, which can make it difficult to scale ml algorithms.
Latency: real-time or near-real-time decision-making is crucial.
Interoperability: ml algorithms may be challenging to integrate into iot devices and systems because they are frequently created using various technologies and standards.
Energy efficiency: iot devices often have limited power and processing resources, which can make it difficult to run complete machine learning algorithms.
Security: iot devices could be exposed to security risks like viruses or hacking.
What is the role of cloud computing in IoT?
cloud computing and iot complement each other. Cloud computing allows iot devices to record, capture, process and store data at a massive scale.
What are the benefits and limitations of edge computing?
by edge computing, we can better control data, reduce cost, provide faster insights and actions, and enable more continuous and streamlined operations. A very good option for sensitive areas - sensitive data. The limitation with edge is that large, complex models can not be deployed to edge service.
Edge vs cloud
Cloud Computing uses centralized servers in large, remote data centers to process and analyze data. It is best for applications that don’t need instant responses.
Edge Computing is a distributed system closer to users and devices. It processes data locally and analyzes it in real-time, making it ideal for situations where low latency is important and every millisecond counts.
Summary:
• Cloud: Good for non-time-sensitive tasks, data stored remotely. • Edge: Best for real-time tasks, data processed locally.
Cloud computing
Cloud computing is a model that provides users with easy and immediate access to shared resources via the internet. These resources can include networks, servers, storage and applications. Users can quickly access and configure these resources with minimal management and without needing much interaction with the service provider. Instead of saving files on a dedicated hard disk or local storage device, cloud-based storage makes it possible to save remotely. Five key characteristics of cloud computing are, On-Demand Self-Service, Ubiquitous Network Access, Resource Pooling, Rapid Elasticity and Measured Service.
What is MLOps?
the process of automating and productionalizing machine learning applications and workflow. A methodology used to implement and deploy ML in IoT. It enables faster model development and deployment by automating key steps such as monitoring, validating and re-training models. Continuous integration, deployment, and traning.
Which are the elements for building ML systems?
configuration, automation, data collection, data verification, feature engineering, ML code, testing and debugging, model analysis, process management, metadata management, serving infrastructure, and monitoring.
What is the ML life cycle and what challenges are there in the cycle?
The different steps required to develop and implement a machine learning model, from data collection to deployment.
Business understanding → data collection → data analysis → data processing → modelling → model evaluation and testing → model analyzing → trained model → repository → model deployment.
Challenges: Time-consuming, it is manual, not reusable, error-prone.
Clustering
Identification of groups with similar characteristics, dividing data into groups (or ‘clusters’) where the items within each group are more similar to each other than they are to the items in other groups.
Edge computing
Edge Computing processes data close to where it’s created (e.g., sensors, devices, offices) instead of sending it to a central data center. This reduces latency, increases efficiency and handles large amounts of data from sensors and IoT devices. A local gateway can handle certain applications without needing to send all data to the cloud or a central server.
Benefits:
* Real-time insights * Faster decision-making * Only important data is sent to the central data center for more analysis.
Why cloud computing
Cloud Computing uses advanced networking, storage, and processing to provide hardware and software resources managed together. This makes security, resource management, and fault tolerance simpler. It’s mainly used for business computing and has a big economic impact.
Key benefits:
• Appears to offer unlimited resources for easy scaling • Pay-as-you-go pricing means no upfront costs
Give examples of two cloud computing services
AWS and Azure
Different cloud computing types
Private, Hybrid, Community and Public
Describe the private cloud computing type
used for a singel organisation, can be internally or externally hoster. It may be managed by the organization or a theird party, may exist on or off premise. E.g., Jetstream, RedCloud. Pros: may be cheaper, you can keep it off the Internet so data can be safe, optimize your own hardware, control everything. Cons: you are responsible for everything, not as many high-level services may not be cheaper, you manage physical and system security.
Describe the hybrid cloud computing type
Hybrid Cloud is a mix of two or more clouds (private and/or public) that stay separate but work together. It combines the advantages of different cloud types and can be hosted both internally and externally.
Used by large companies when:
• Shifting workloads from on-premises to the cloud • Legacy apps can’t move to the cloud • Licensed apps need to stay on-premises • Data must be kept secure on-premises
Describe the community cloud computing type
shared by several organizations and supports a specific community that has shared concernes, typically externally hoster, but can be internally hoster by one of the organizations. It may be managed by the organization or a theird party, my exist on or off premise. cloud computing platform. E.g., Banking and research sectors.
Describe the public cloud computing type
provisioned for open use for the public or a large industry group by a particular organization who also hosts the service and selling the cloud service. Use of the public aws, Microsoft Azure. Pros: Massive scale, Huge and growing lists of service, security is strong, highly competitive on pricing due to economies scale, hardware constantly upgraded. Cons: rules prohibit data moving to the cloud, funding models may make it hard to use, fear of “vendor lock-in”.
Examples on delivery models in cloud computing
SaaS (software as a service), PaaS (platform as a service), IaaS (infrastructure as a service).
Describe the delivery model IaaS
This service provides essential IT resources like servers, networking, and storage, managed by a provider. Customers can use these resources without dealing with the physical aspects of data centers, such as managing temperature or maintenance. The resources are shared, scalable, and perfect for situations where demand changes quickly or when companies don’t want to buy their own hardware.
Benefits:
• Full control over computing power and resources • Flexible and easy to scale up or down • Portable and easy to maintain
Cons:
• Possible security risks • Challenges with disaster recovery • Risk of using outdated virtual machines without updates
Describe the delivery model SaaS
SaaS (Software as a Service) is ready-to-use software provided and managed by a third party. Users pay to access it, so they don’t need to worry about managing the software or infrastructure behind it. SaaS is great for applications like email and other services that don’t need infrastructure management. However, it’s not the best for real-time applications or when strict data hosting rules are needed.
Best uses for SaaS:
• Apps used often (e.g., email) • Apps with peak usage periods (e.g., billing systems) • Apps needing web or mobile access (e.g., sales tools)
Advantages:
• No need to manage software licenses or maintenance • Automatic updates and simplified administrative tasks
Disadvantages:
• Security risks: data stored in the cloud • Latency: cloud storage can cause delays in processing data • Internet dependency: requires a stable connection to work
What are the barriers to cloud computing
- Service Availability: What happens if the provider can’t deliver? This can disrupt access to services.
- Vendor Lock-In: Once using one cloud provider, it’s hard to switch to another.
- Data Confidentiality and Auditability: Keeping data private and ensuring it can be checked securely.
- Data Transfer Bottlenecks: Slow networks can delay moving large amounts of data, causing issues for data-heavy apps.
- Performance Unpredictability: Sharing resources can lead to inconsistent performance.
- Elasticity: New algorithms are needed to manage resource allocation and workload placement.
Give example of other delivery models
DBaaS (database as a service) and FaaS (function as a service).
Why are cloud services needed in IoT systems?
- connecting things (i.e., device and objects),
- establish communication channels,
- collect data,
- storing data,
- analyse data (including ML),
- Enable things to operate even without internet connectivity; When connections are possible, they can connect to report data and status.
What is a IoT system?
a IoT system works through the real-time collection and exchange of data. Has three components: Smart devices, IoT applications and a graphical user interface. E.g. of IoT systems is connected cars, connected homes and smart cities.
Describe the delivery model PaaS -
PaaS (Platform as a Service) provides a platform managed by the cloud provider, which includes resources like the operating system, runtime, and middleware. This allows developers to focus on building and managing their code, logic, and data without worrying about platform maintenance. Best uses for PaaS:
Collaborative software development! Automated app deployment and testing. Not ideal for: Apps that need to be moved easily between environments. Apps using proprietary programming languages. Projects requiring custom hardware.
Advantages:
* Ability to update and modify the operating system * Easier global collaboration * Lower overall costs by reducing hardware expenses
Disadvantages:
* Security risks: some providers may not allow you to move software off their platform * Limited flexibility for some custom needs
Edge computing serves four primary usage patterns, which?
- Reduced latency: Edge systems can be placed closer to end users and services
(less network hops and propagation).
• Bandwidth preservation: It supports filtering, caching, and data compression
techniques to efficiently maximize available bandwidth.
• Resilient computing: When communication is not reliable, IoT systems must
store/cache data at the edge until communication is restored.
• Security and privacy: It help to secure or even remove certain data before it
travels further into the cloud or other edges.
