How to Deploy to Kubernetes using Argo CD and GitOps

Using Kubernetes to deploy your application can provide significant infrastructural advantages, such as flexible scaling, management of distributed components, and control over different versions of your application. However, with that increased control comes increased complexity. Continuous Integration and Continuous Deployment (CI/CD) systems usually work at a high level of abstraction in order to provide version control, change logging, and rollback functionality. A popular approach to this abstraction layer is called GitOps.

GitOps, as originally proposed by Weaveworks in a 2017 blog post, uses Git as a “single source of truth” for CI/CD processes, integrating code changes in a single, shared repository per project and using pull requests to manage infrastructure and deployment.

There are several tools that use Git as a focal point for DevOps processes on Kubernetes. In this tutorial, you will learn to use Argo CD, a declarative Continuous Delivery tool. Argo CD provides Continuous Delivery tooling that automatically synchronizes and deploys your application whenever a change is made in your GitHub repository. By managing the deployment and lifecycle of an application, it provides solutions for version control, configurations, and application definitions in Kubernetes environments, organizing complex data with an easy-to-understand user interface. It can handle several types of Kubernetes manifests, including Jsonnet, Kustomize applications, Helm charts, and YAML/json files, and supports webhook notifications from GitHub, GitLab, and Bitbucket.

Step 1 — Installing Argo CD on Your Cluster

In order to install Argo CD, you should first have a valid Kubernetes configuration set up with kubectl, from which you can ping your worker nodes. You can test this by running kubectl get nodes:

kubectl get nodes

This command should return a list of nodes with the Ready status:

Output
NAME                   STATUS   ROLES    AGE   VERSION
pool-uqv8a47h0-ul5a7   Ready    <none>   22m   v1.21.5
pool-uqv8a47h0-ul5am   Ready    <none>   21m   v1.21.5
pool-uqv8a47h0-ul5aq   Ready    <none>   21m   v1.21.5

If kubectl does not return a set of nodes with the Ready status, you should review your cluster configuration and the Kubernetes documentation.

Next, create the argocd namespace in your cluster, which will contain Argo CD and its associated services:

kubectl create namespace argocd

After that, you can run the Argo CD install script provided by the project maintainers.

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Once the installation completes successfully, you can use the watch command to check the status of your Kubernetes pods:

watch kubectl get pods -n argocd

By default, there should be five pods that eventually receive the Running status as part of a stock Argo CD installation.

Output
NAME                                  READY   STATUS    RESTARTS   AGE
argocd-application-controller-0       1/1     Running   0          2m28s
argocd-dex-server-66f865ffb4-chwwg    1/1     Running   0          2m30s
argocd-redis-5b6967fdfc-q4klp         1/1     Running   0          2m30s
argocd-repo-server-656c76778f-vsn7l   1/1     Running   0          2m29s
argocd-server-cd68f46f8-zg7hq         1/1     Running   0          2m28s

You can press Ctrl+C to exit the watch interface. You now have Argo CD running in your Kubernetes cluster! However, because of the way Kubernetes creates abstractions around your network interfaces, you won’t be able to access it directly without forwarding ports from inside your cluster. You’ll learn how to handle that in the next step.

Step 2 — Forwarding Ports to Access Argo CD

Because Kubernetes deploys services to arbitrary network addresses inside your cluster, you’ll need to forward the relevant ports in order to access them from your local machine. Argo CD sets up a service named argocd-server on port 443 internally. Because port 443 is the default HTTPS port, and you may be running some other HTTP/HTTPS services, it’s common practice to forward those to arbitrarily chosen other ports, like 8080, like so:

kubectl port-forward svc/argocd-server -n argocd 8080:443

Port forwarding will block the terminal it’s running in as long as it’s active, so you’ll probably want to run this in a new terminal window while you continue to work. You can press Ctrl+C to gracefully quit a blocking process such as this one when you want to stop forwarding the port.

In the meantime, you should be able to access Argo CD in a web browser by navigating to localhost:8080. However, you’ll be prompted for a login password which you’ll need to use the command line to retrieve in the next step. You’ll probably need to click through a security warning because Argo CD has not yet been configured with a valid SSL certificate.

Before using it, you’ll want to use kubectl again to retrieve the admin password which was automatically generated during your installation, so that you can use it to log in. You’ll pass it a path to a particular JSON file that’s stored using Kubernetes secrets, and extract the relevant value:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

Output
fbP20pvw-o-D5uxH

You can then log into your Argo CD dashboard by going back to localhost:8080 in a browser and logging in as the admin user with your own password:

Once everything is working, you can use the same credentials to log in to Argo CD via the command line, by running argocd login. This will be necessary for deploying from the command line later on:

argocd login localhost:8080

You’ll receive the equivalent certificate warning again on the command line here, and should enter y to proceed when prompted. If desired, you can then change your password to something more secure or more memorable by running argocd account update-password. After that, you’ll have a fully working Argo CD configuration. In the final steps of this tutorial, you’ll learn how to use it to actually deploy some example applications.

Deploying Your First Application with ArgoCD

Step 1: Create a Git Repository

First, create a Git repository that contains your Kubernetes manifests. This repository will serve as the source for ArgoCD to deploy your application.

Step 2: Create an Application in ArgoCD

Method 1: Using the ArgoCD UI

1. Create a Git Repository: First, create a Git repository that contains your Kubernetes manifests. This repository will serve as the source for ArgoCD to deploy your application.
2. Create an Application in ArgoCD:

• In the ArgoCD dashboard, click on New App and fill in the following details:
• Application Name: my-first-app
• Project: default
• Sync Policy: Manual or Automatic (your choice)
• Repository URL: URL of your Git repository
• Path: Path to your Kubernetes manifests in the repository
• Cluster URL: Leave as default for your current cluster
• Namespace: The namespace where the application should be deployed

Step 1: ArgoCD Application Creation

Step 2: ArgoCD Application Creation

Once the application is created, click on Sync to deploy the application to your Kubernetes cluster. If Sync Policy Set to Automatic then application will be deployed to kubernetes automatically.

Syncing and Monitoring Your Application

After syncing, ArgoCD will start deploying the application. You can monitor the progress in the dashboard. If there are any issues, ArgoCD will highlight them, and you can take corrective actions directly from the UI.

ArgoCD also supports features like automated rollbacks, self-healing, and more, ensuring your Kubernetes cluster is always in the desired state as defined in your Git repository.

Step 6.1: Automatic Sync on GitHub Updates

One of the powerful features of ArgoCD is its ability to automatically sync your Kubernetes cluster with your Git repository. This means that any changes you make to your manifests in GitHub will be automatically applied to your cluster.

Jenkins Scripted Pipeline - Create Jenkins Pipeline for Automating Builds, Code quality checks, Deployments to Tomcat - How to build, deploy WARs using Jenkins Pipeline - Build pipelines integrate with github, Sonarqube, Slack, JaCoCo, Nexus, Tomcat

 

Jenkins Scripted Pipeline - Create Jenkins Pipeline for Automating Builds, Code quality checks, Deployments to Tomcat - How to build, deploy WARs using Jenkins Pipeline - Build pipelines integrate with github, Sonarqube, Slack, JaCoCo, Nexus, Tomcat

What are Pipelines in Jenkins?

- Pipelines are better than freestyle jobs, you can write a lot of complex tasks using pipelines when compared to Freestyle jobs.

- You can see how long each stage takes time to execute so you have more control compared to freestyle.

- Pipeline is groovy based script that have set of plug-ins integrated for automating the builds, deployment and test execution.

- Pipeline defines your entire build process, which typically includes stages for building an application, testing it and then delivering it. 

 - You can use snippet generator to generate pipeline code for the stages you don't know how to write groovy code.

- Pipelines are two types - Scripted pipeline and Declarative pipeline. Click here to know the difference between them.

Pre-requisites:

Install plug-ins
1. Install Deploy to container, Slack, Jacoco, Nexus Artifact Uploader and SonarQube plug-ins (if already installed, you can skip it)

Steps to Create Scripted Pipeline in Jenkins

1. Login to Jenkins

2. Create a New item

3. Give name as MyfirstPipelineJob and choose pipeline

4. Click ok. Pipeline is created now

5. Under build triggers, click on poll SCM, schedule as

H/02 * * * *

6. Go to Pipeline definition section, click on Pipeline syntax link. under sample step drop down, choose checkout: Checkout from version control. enter bitbucket or GitHub Repository URL, and enter right credentials. Click here to learn to use PAT if you are using GitHub. scroll down, click on Generate Pipeline script. Copy the code.

7. Now copy the below pipeline code highlighted section into Pipeline section in the pipeline. Please copy stage by stage

8. Change Maven3, SonarQube variables and also Slack channel name as highlighted above in red as per your settings.

9. For Nexus Upload stage, You need to change the Nexus URL and credentials ID for Nexus (which you can grab from Credentials tab after login)

10. For Dev Deploy stage, you can copy credentials ID used for connecting to Tomcat.

Pipeline Code:

node {

    def mvnHome = tool 'Maven3'
    stage ("checkout")  {
       copy code here which you generated from step #6
    }

   stage ('build')  {
    sh "${mvnHome}/bin/mvn clean install -f MyWebApp/pom.xml"
    }

     stage ('Code Quality scan')  {
       withSonarQubeEnv('SonarQube') {
       sh "${mvnHome}/bin/mvn -f MyWebApp/pom.xml sonar:sonar"
        }
   }
  
   stage ('Code coverage')  {
       jacoco()
   }

   stage ('Nexus upload')  {
        nexusArtifactUploader(
        nexusVersion: 'nexus3',
        protocol: 'http',
        nexusUrl: 'nexus_url:8081',
        groupId: 'myGroupId',
        version: '1.0-SNAPSHOT',
        repository: 'maven-snapshots',
        credentialsId: '2c293828-9509-49b4-a6e7-77f3ceae7b39',
        artifacts: [
            [artifactId: 'MyWebApp',
             classifier: '',
             file: 'MyWebApp/target/MyWebApp.war',
             type: 'war']
        ]
     )
    }
   
   stage ('DEV Deploy')  {
      echo "deploying to DEV Env "
      deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

    }

  stage ('Slack notification')  {
    slackSend(channel:'channel-name', message: "Job is successful, here is the info -  Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")
   }

   stage ('DEV Approve')  {
            echo "Taking approval from DEV Manager for QA Deployment"     
            timeout(time: 7, unit: 'DAYS') {
            input message: 'Do you approve QA Deployment?', submitter: 'admin'
            }
     }

stage ('QA Deploy')  {
     echo "deploying into QA Env " 
deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

}

  stage ('QA notify')  {
    slackSend(channel:'channel-name', message: "QA Deployment was successful, here is the info -  Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")
   }

stage ('QA Approve')  {
    echo "Taking approval from QA manager"
    timeout(time: 7, unit: 'DAYS') {
        input message: 'Do you want to proceed to PROD Deploy?', submitter: 'admin,manager_userid'
  }

}

stage ('PROD Deploy')  {
     echo "deploying into PROD Env " 
deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

}
}

11. Click Apply, Save

12. Now click on Build. It should execute all the stages and show pipeline view like this.

Jenkins Nexus Integration - How to integrate Nexus with Jenkins

 

Jenkins Nexus Integration - How to integrate Nexus with Jenkins

 

You need to install Nexus Artifact Uploader plug-in to integrate Nexus with Jenkins. Let us see how to integrate Jenkins with Nexus and able to upload War/Ear/Jar/Exe/DLLs.


Pre-requistes:

Install Nexus Artifact Uploader plugin in Jenkins.

Steps:

1. Once you installed the above plug-ins, click existing FreeStyle job configuration or create a new job.

2. Under source code management. enter bitbucket repo url and git credentials.

3. Go to build section, add Maven targets. with goals clean install. Also click advance, give path of POM.xml

4. click on Add build step, choose Nexus artifact uploader.

6. Click on Apply, Save.

7. Now login to Nexus repo, Click on Components, Click on maven-snapshots

8. you shall see the WAR being uploaded here.

How to Install Nexus on RedHat Linux

 

How to Install Nexus on RedHat Linux

Nexus is binary repository manager, used for storing build artifacts. We will eventually integrate Nexus with Jenkins for uploading WAR/EAR/JAR files there.

Here are the steps for installing Sonatype Nexus 3 in RHEL in EC2 on AWS. Please create a new Redhat EC2 instance with small type. Choose Redhat Enterprise 8.

Pre-requisites:
Make sure you open port 8081 in AWS security group

Installation Steps:

sudo yum install wget -y

Download Open JDK

sudo yum install java-1.8.0-openjdk.x86_64 -y

Execute the below command to navigate to /opt directory by changing directory:
cd /opt

Download Nexus
sudo wget http://download.sonatype.com/nexus/3/nexus-3.23.0-03-unix.tar.gz

Extract Nexus
sudo tar -xvf nexus-3.23.0-03-unix.tar.gz
sudo mv nexus-3.23.0-03 nexus

Create a user called Nexus
sudo adduser nexus

Change the ownership of nexus files and nexus data directory to nexus user.
sudo chown -R nexus:nexus /opt/nexus

sudo chown -R nexus:nexus /opt/sonatype-work

Configure to run as Nexus user

change as below screenshot by removing # and adding nexus
 sudo vi /opt/nexus/bin/nexus.rc

Modify memory settings in Nexus configuration file
sudo vi /opt/nexus/bin/nexus.vmoptions

Modify the above file as shown in red highlighted section:

-Xms512m
-Xmx512m
-XX:MaxDirectMemorySize=512m

after making changes, press wq! to come out of the file.

Configure Nexus to run as a service

sudo vi /etc/systemd/system/nexus.service
Copy the below content highlighted in green color.

[Unit]

Description=nexus service

After=network.target

[Service]

Type=forking

LimitNOFILE=65536

User=nexus

Group=nexus

ExecStart=/opt/nexus/bin/nexus start

ExecStop=/opt/nexus/bin/nexus stop

User=nexus

Restart=on-abort
[Install]

WantedBy=multi-user.target

Create a link to Nexus
sudo ln -s /opt/nexus/bin/nexus /etc/init.d/nexus

Execute the following command to add nexus service to boot.

sudo chkconfig --add nexus
sudo chkconfig --levels 345 nexus on

Start Nexus
sudo service nexus start

Check whether Nexus service is running
sudo service nexus status

Check the logs to see if Nexus is running
tail -f /opt/sonatype-work/nexus3/log/nexus.log

You will see Nexus started..
If you Nexus stopped, review the steps above.

Now press Ctrl C to come out of this windows.

Once Nexus is successfully installed, you can access it in the browser by URL - http://public_dns_name:8081

Click on Sign in link

user name is admin and password can be found by executing below command:

sudo cat /opt/sonatype-work/nexus3/admin.password

Copy the password and click sign in.
Now setup admin password as admin123

you should see the home page of Nexus: