ECS Jenkins Plugin to create ephemeral Slaves in Fargate

In the following article I will show you how to configure Jenkins ECS plugin https://github.com/jenkinsci/amazon-ecs-plugin to create Slaves in Fargate through a JenkinsFile. We will use a pipeline with Jenkinsfile to have as much configuration there as posible. I choose Fargate to avoid having to manage ECS nodes (but remember that Fargate does not allow to have permanent volumes).

About Jenkins:

I will assume it is in a EC2.

You can install it form ansible-galaxy, Helm,… good to do like this because you can easily install Jenkins plugins in the future without hardcoding it through web interface.

Jenkins will have communication with ECS tasks, for example: be in same VPC and same subnet.

IAM Role: the one defined here https://plugins.jenkins.io/scalable-amazon-ecs/

You will need to install in your Jenkins the plugin: https://plugins.jenkins.io/scalable-amazon-ecs/

Create a Git token so that Jenkins pipeline can checkout the repo in the Slave.

Mange Jenkins →Nodes administration →Configure clouds:

Jenkins Pipeline:

Jenkinsfile that we will use in the pipeline:

pipeline {
agent none
   stages {
       stage('Test') {
           agent {
               ecs {
                  cloud 'jenkins-slave-ecs'
                  image 'XXXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/jenkins-slave-ansible:1.0'
                  launchType 'FARGATE'
                  memory 1024
                  cpu 256
                  subnets('subnet-08086cbe2d97a1ff1')
                  securityGroups('sg-08f3f54702fb3992e')
                  taskrole 'arn:aws:iam::XXXXXXXXX:role/ecsTaskExecutionRole'
                  executionRole 'arn:aws:iam::XXXXXXXXX:role/ecsTaskExecutionRole'
                  assignPublicIp true
               }
           }
           steps {
               sh 'ansible -h'
           }
       }
   }
}

‍

IAM ecsTaskExecutionRole will have in addition, permissions to read from ECR.

About the Dockerfile , you can integrate the JNLP (Jenkins slave software) into your own image, I have done with Python like this:

ROM jenkins/jnlp-slave:alpine as jnlp
FROM python:alpine
RUN apk -U add openjdk8-jre
COPY — from=jnlp /usr/local/bin/jenkins-agent /usr/local/bin/jenkins-agent
COPY — from=jnlp /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
ENTRYPOINT [“/usr/local/bin/jenkins-agent”]

You will also need to have a ECS Cluster, Fargate type, with name “jenkins-slave-ecs”, same than in Jenkinsfile.

More info about JNLP: https://hub.docker.com/r/jenkinsci/jnlp-slave/

Additional info about ECS plugin parameters for the Jenkinsfile: https://jenkins.io/doc/pipeline/steps/amazon-ecs/

Schemmas of the architecture: https://www.slideshare.net/PhilippKoch11/never-stand-in-row-again-with-jenkins-on-aws-ecs-and-fargate