CICD : Code items Flashcards
General info
CodeCommit
Codepipeline
Codebuild
Codedeploy
Commit: Store Code
Pipeline: orchestration of all events of build test and deploy
Build: Building and testing code
Deploy: Deploy code to Ec2 fleet NOT BEANSTALK
Code Commit Version Control
Version control: Understand changes over time and have ability to roll back.
Version control system is required, EX : GIT
Git Respository lives on machine or online: Allows
Collaboration
Backup
Viewable and auditable.
Code Commit Git repository
Examples:
Github
Bitbucket etc
AWS CODECOMMIT**** Private No size limit Managed and available highly Code ONLY VIA AWS CLOUD ACCOUNT (secure) Secure (Encrypted , has access control) Integrated with Jenkins, Codebuild, CI tools
Code Commit SECURITY
Interactions are done using Git (standard commands for Git etc)
Authentication in Git:
- SSH Keys: IAM console generation
- HTTPS: CLI authentication helper
- MFA: extra auth
Authorization in Git:
-IAM policy manager user/role rights to repo
Encryption:
- Automatically done via KMS
- Https for encryption in transit
Cross account access:
- Dont share SSH keys
- Dont share aws creds
- Use IAM role, other person uses STS for cross account access assumerole* to become role.
CODECOMMIT VS GITHUB
Similar: Both Git Repo Both have code pull request capability Both can be integrated into Codebuild Both support HTTPS and SSH for authentication
Differences: Security: -Github uses github users -Codecommit: IAM users and roles Hosting: -Github online, or on Github enterprise on own server -Codecommit: Manage and hosted by AWS
Code Commit Notifications
Trigger VIA
AWS SNS
AWS LAMBDA
AWS Cloudwatch event rules
SNS/LAMBDA (actions of adding /delete code)
- Deletion of branches
- push detection on master branch
- notify external build system
- lambda code analysis (see if credentials are in code!)
Cloudwatch event rules (more around pull request)
- Trigger for pull request update (Create, update, delete, comment)
- Commit then comments, then event is triggered by COMMENT**
- This goes to SNS topic
CODEPIPLINE OVERVIEW
Visual tool to perform continuous delivery, orchestrate between sources and builds
Source: Github/codecommit/S3
Build: Codebuild , jenkins, etc
Load testing: 3rd party tool
Deploy: AWS codedeploy/ beanstalk/ cloudformation /ecs
Made of Stages
Each stage has sequential actions or paralell.
Build>Test>Deploy>load test>ETC
Can have manual approval at any stage!
TECH STACK CODE BUILD TEST DEPLOY PROVISION
CODE - Code Commit / github
BUILD- Codebuild /jenkins
TEST- Codebuild /jenkins
DEPLOY - Elastic beanstalk / Codedeploy(ec2)
PROVISION - elastic beanstalk / EC2 via cloudformation and previously codedeploy
ORCHESTRATION OF ALL IS CODEPIPELINE
CODEPIPLINE and artifacts
Pipeline has stages, each stage can create artifacts
Artifacts are created by one stage and used by another
These artifacts are stored in S3 and passed into next stage.
Codecommit: Store code into S3
Codebuild: compile and test code from S3 and place result in s3
Deploy: Load s3 code and start ec2 deployment.
CODEPIPLINE Troubleshooting
Cloudpipeline state changes
- state change in pipeline will generate AWS CLOUDWATCH EVENT which can create SNS notification.
useful for failure in failed pipeline
can be used for when someone cancels event.
- Failure, causes pipeline to stop, you can get information at console. SNS notification should be used to prompt us to look into console
AWS API calls for this can be seen in cloudtrail.
**PERMISSIONS ISSUE: If pipeline cannot perform an action it means that attached “IAM service Role” doesn’t have enough permissions in IAM policy.
Codebuild Overview
security
logging
docker
Building and testing application
Like Jenkins
Manged Build Service, Alternative to Jenkins
Continuous Scaling
Pay for usage
Leverage Docker for reproducible builds
May be extended and can use our own Docker images for codebuild, allows customization
SECURE:
integration with KMS for build artifacts
integration with IAM for build permissions, can run in own VPC, can use cloudtrail for call logging.
Codebuild Overview Capabilities code from Build instructions Output logs alarms for failure BUILDS CAN BE DEFINED:
Process Source Code from:
Github, Codecommit, codepipeline, s3
Build instructions can be defined in code (BUILDSPEC.YML)
Output LOGS to Amazon s3 and AWS cloudwatch logs
after build finish, container disappears, this log remains
Metrics: Cloudwatch alarms for failed build and notification
Cloud events / aws lambda allows integration for event reporting, USE SNS for notifications
ABLE to reproduce cloudwatch on local PC for troubleshooting.
BUILDS CAN BE DEFINED:
IN codepipeline
or codebuild itself.
CODEBUILD: VPC
By Default Codebuild containers are launched outside VPC, so it needs to be given access to reach resources inside your VPC
Dont Via VPC configuration: allows access if group rules allow it. Security group**
VPC ID
Subnet ID
Security Group ID
RDS, elasticache, Ec2, ALB etc can now be accessed
CODE DEPLOY OVERVIEW
for what?
what else does it
what does this do that’s different.
-Ability to launch to many EC2 instances
-NOT managed by elastic Beanstalk
-Several ways to handle deployments
chef, terraform, puppet etc
-DEPLOYMENT IN THIS CASE: Will be done via AWS codedeploy! managed service
CODE DEPLOY: Setup
What needs to be done before this process can work
What message is sent to machines
where are files taken from for deployment
how is it reported back?
- Each EC2 or premise machine, MUST be running Code DEPLOY AGENT!
- Agent polls for AWS codedeploy service
- Source code is bundled with Appspec.yaml file
- This code is given to repo (github or s3) and also is able to TRIGGER DEPLOYMENT
- Triggered deployment reaches Ec2 and DEPLOY AGENT
- Deploy agent Downloads code with appspec.yaml file
- Ec2 will run based on appspec.yaml file which is in root of code
- Codedeploy will report of success/failure of deployment
CODE DEPLOY: Considerations
How to organize
what tools can be used
do i need to provision, or does it happen on its own?
Ec2 instances grouped by deployment group:
Dev/prod/test. this specification is flexible
CODEDEPLOY can be chained to CodePipeline to use artifacts from S3 in deployment.
Can reuse existing setup tools, can work with ASG/pre launched EC2/ALB. Flexible to be used on various platforms
Can be done blue/green, ONLY VIA EC2
Support with AWS lambda deployment
DOES NOT PROVISION RESOURCES
CODE DEPLOY: Parts
Application: Unique name
Compute platform: Ec2/ on premise
Deployment configuration: Deployment rules for success / failure
—-Ec2 / on premise: Can specify min number of healthy instances for deployment.
—-Lambda: How traffic routed to lambda functions
Deployment Group: Groups tagged, Gradual deployment
Deployment Type: development Blue/green deploy
IAM instance profile: need to give EC2 permissions to pull from repo s3 or github
Application revision: Code + Appspec.yml
Service role: Role for Codedeploy to access for function
Target revision: Target deployment application version. Where you want the target version of application when upgraded
CODE DEPLOY:APPSPEC APPSPEC.YAML
File section:
Hooks:
File section: How to source and copy from Repo, S3 or Github
Hooks: Set of instructions on how to deploy new versions, can have timeouts for scripting. ORDER IS:
ApplicationStop: Stop application to deploy this version
Downloadbundle: download from repo
Beforeinstall: prep before installation
afterinstall: cleanup after install/ launch server etc
applicationstart: start new code
validateservice: HEALTH CHECK, how to see if application started correctly.
Beforeallowtraffic
allowtraffic
afterallowtraffic
CODE DEPLOY: Deployment config
Configs:
One at a time, if one instance fails then it all stops
Half at a time
All at once: can have no healthy hosts, ok for Dev
custom: ex: have minimum healthy hosts to be set around 75%
Failures:
instances stay in failed state
NEW DEPLOYMENTS WILL BE DEPLOYED TO FAILED STATES FIRST!
rollback: deploy old deployment or enable automated rollback
Deployment targets:
Set of ec2 instances with tags (prod/dev/etc)
or can deploy to ASG
OR COMBINE: Tags with ASG for deployment segments
Customization in scripts with DEPLOYMENT_GROUP_NAME environment variables.
CODE DEPLOY:
EC2
(IN PLACE DEPLOYMENT)
Define how deploy:
APPSPEC.YAML + Deployment strategy
Will need to do in-place update to your fleet of ec2
Can use hooks to verify deployment after each phase
(IN PLACE DEPLOYMENT)
- All v1
- Half is taken down
- taken down is upgraded to v2
- do the same with the other half.
CODE DEPLOY: ASG (IN PLACE DEPLOYMENT) (AUTOMATED DEPLOYMENT FOR NEW) (BLUEGREEN DEPLOYMENT)
ASG do both
In-place/blue-green
- In place:
Update using current Ec2
New created instances will get automatic deployment - Blue-green
New ASG created, settings are copied
choose how long to keep old instances
Must be done via a ELB
2A. diagram
ASG exists, with one group
another group created via ASG and settings copied
ELB balances between the two groups
Code deploy will deploy to new location
timer will keep old instances until your set time passes
new instances will hold traffic
CODE DEPLOY:
ROLLBACK
- Current version exists, but previous version is needed because of error
- AUTOMATED rollback options bay be enabled
- -for failure
- -roll back when cloud-watch alarms are met
- -can be disabled also
ROLLBACK: Codedeploy redeploys last known good revision as a NEW DEPLOYMENT, this gets a new version ID!!!!
CODESTAR
Groups CICD SERVICES:
Github/codecommit/codebuild/codedeploy/cloudformation
codepipeline/cloudwatch
Creates one stop dashboard to view all components: gives insights to all.
Free service, pay for usage of other services
limited customization
Help create CICD ready projects for Ec2/ Lambda/ Beanstalk
Supports many languages
Issue tracking integration: JIRA/Github issues
ability to integrate with Cloud9 to obtain web IDE (a web environment to edit code) not all regions
Code build
Cache
Buildspec.yaml
Downloading dependencies is a critical phase in the build process. These dependent files can range in size from a few KBs to multiple MBs. Because most of the dependent files do not change frequently between builds, you can noticeably reduce your build time by caching dependencies.
Note:The cache section in the build specification instructs AWS CodeBuild about the paths to be cached. Like theartifacts section, the cache paths are relative to$CODEBUILD_SRC_DIR and specify the directories to be cached. In this example, Maven stores the downloaded dependencies to the /root/.m2/ folder, but other tools use different folders. For example, pip uses the /root/.cache/pip folder, and Gradle uses the /root/.gradle/caches folder. You might need to configure the cache paths based on your language platform.
