Using a Different System?
-
How to Install and Configure Concourse CI on Ubuntu 16.04
Introduction
Continuous Integration is a DevOps software development practice which enables the developers to frequently merge the modified code into the shared repository many times a day. After each merge, automatic builds and tests are performed to detect problems in the code. It enables the developers to find and resolve the errors quickly to improve software quality and provide continuous delivery of the software. Switching to and fro from Concourse is very easy as it keeps all its configuration in declarative files that can be checked into version control. It also provides a web user interface which displays the build information interactively.
Concourse Components.
- ATC is the main component of the Concourse. It is responsible for running the Web UI and API. It also takes care of all the pipeline scheduling.
- TSA is a custom built SSH server. It is responsible for securely registering a worker with ATC.
- Workers further runs two different services:
- Garden is a container runtime and an interface for orchestrating containers remotely on a worker.
- Baggageclaim is a cache and artifact management server.
- Fly is a command line interface used to interact with the ATC to configure Concourse Pipelines.
Prerequisites
- A Vultr CentOS 7 server instance.
- A sudo user.
Be sure to replace all occurrences of 192.0.2.1
and ci.example.com
with your actual Vultr public IP address and actual domain name.
Update your base system using the guide How to Update CentOS 7. Once your system has been updated, proceed to install PostgreSQL.
Install and Configure PostgreSQL Database
PostgreSQL is an object relational database system. Concourse stores its pipeline data into a PostgreSQL database.
Add the PostgreSQL repository.
sudo rpm -Uvh https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm
Install the PostgreSQL database server.
sudo yum -y install postgresql96-server postgresql96-contrib
Initialize the database.
sudo /usr/pgsql-9.6/bin/postgresql96-setup initdb
initdb
creates a new PostgreSQL database cluster, which is a collection of databases that are managed by a single server instance. Edit the pg_hba.conf
file to enable MD5 based authentication.
sudo nano /var/lib/pgsql/9.6/data/pg_hba.conf
Find the following lines and change the values peer
and ident
in the METHOD
column to trust
and md5
, respectively.
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all peer
# IPv4 local connections:
host all all 127.0.0.1/32 ident
# IPv6 local connections:
host all all ::1/128 ident
Once updated, the configuration should look like this.
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
Start the PostgreSQL server and enable it to start automatically at boot time.
sudo systemctl start postgresql-9.6
sudo systemctl enable postgresql-9.6
Change the password for the default PostgreSQL user.
sudo passwd postgres
Login as PostgreSQL user:
sudo su - postgres
Create a new PostgreSQL user for Concourse CI.
createuser concourse
Note: The default PostgreSQL user can be used for authentication of the database, but it is recommended to use a dedicated user for authentication of Concourse database in a production setup.
PostgreSQL provides a shell to run queries on the database. Switch to the PostgreSQL shell by running:
psql
Set a password for the newly created Concourse database user.
ALTER USER concourse WITH ENCRYPTED password 'DBPassword';
Important: Replace DBPassword
with a strong password. Make a note of the password as it will be required later in the tutorial.
Create a new database for Concourse.
CREATE DATABASE concourse OWNER concourse;
Exit the psql
shell.
/q
Switch to the sudo user from current postgres user.
exit
Download and Install Concourse CI
Download the latest version of Concourse executable and store it in /usr/bin
so that it can be executed directly. The latest version of Concourse and Fly binaries can be found on the Concourse download page. New releases are very frequent. Replace the link below with the new link for the most recent version.
sudo wget https://github.com/concourse/concourse/releases/download/v3.4.1/concourse_linux_amd64 -O /usr/bin/concourse
Similarly, download the latest version of the fly executable and store it in /usr/bin
.
sudo wget https://github.com/concourse/concourse/releases/download/v3.4.1/fly_linux_amd64 -O /usr/bin/fly
Fly is the command line interface to connect to the ATC API of Concourse CI. Fly is available for multiple platforms such as Linux, Windows and MacOS.
Assign execute permission to the downloaded concourse
and fly
binaries.
sudo chmod +x /usr/bin/concourse /usr/bin/fly
Check if Concourse and Fly are working correctly by checking their version.
concourse -version
fly -version
Generate and Setup RSA Keys
RSA key pairs provide a way to encrypt the communication between the components of the Concourse.
For Concourse to work, at least three pairs of keys must be generated. For encrypting the session data, generate a session_signing_key
. This key will also be used by TSA to sign the requests it makes to the ATC. To secure the TSA SSH server, generate a tsa_host_key
. Finally, generate a worker_key
for each worker.
Create a new directory to store the keys and configuration related to Concourse CI.
sudo mkdir /opt/concourse
Generate the required keys.
sudo ssh-keygen -t rsa -q -N '' -f /opt/concourse/session_signing_key
sudo ssh-keygen -t rsa -q -N '' -f /opt/concourse/tsa_host_key
sudo ssh-keygen -t rsa -q -N '' -f /opt/concourse/worker_key
Authorize the workers’ public key by copying its contents to the authorized_worker_keys
file:
sudo cp /opt/concourse/worker_key.pub /opt/concourse/authorized_worker_keys
Starting Concourse
Concourse provides two separate components which need to be started, the web and the worker.
Start the Concourse web.
sudo concourse web /
--basic-auth-username admin /
--basic-auth-password StrongPass /
--session-signing-key /opt/concourse/session_signing_key /
--tsa-host-key /opt/concourse/tsa_host_key /
--tsa-authorized-keys /opt/concourse/authorized_worker_keys /
--postgres-user=concourse /
--postgres-password=DBPassword /
--postgres-database=concourse /
--external-url http://192.0.2.1:8080
Change the username and password of the basic-auth
if desired. Make sure that the path to the key files are correct and make sure that the correct value for username and password in the PostgreSQL database configuration is provided.
Note: ATC will listen to the default port 8080
and TSA will listen to port 2222
. If authentication is not desired, pass the --no-really-i-dont-want-any-auth
option after removing the basic auth options.
Once the web server is started, the following output should be displayed.
{"timestamp":"1503657859.661247969","source":"tsa","message":"tsa.listening","log_level":1,"data":{}}
{"timestamp":"1503657859.666907549","source":"atc","message":"atc.listening","log_level":1,"data":{"debug":"127.0.0.1:8079","http":"0.0.0.0:8080"}}
Stop the server for now, as a few more things still must be setup.
Start the Concourse CI Worker.
sudo concourse worker /
--work-dir /opt/concourse/worker /
--tsa-host 127.0.0.1 /
--tsa-public-key /opt/concourse/tsa_host_key.pub /
--tsa-worker-private-key /opt/concourse/worker_key
The above command will assume that the TSA is running on localhost and listening to the default port 2222
.
Though the Concourse web and worker can be started easily using the commands above, it is recommended to use Systemd to manage the server.
Configure Environment and Systemd Service
Using Systemd service for managing the application ensures that the application is automatically started on failures and at boot time. The Concourse server does not take data from any configuration file, but it can access the data from environment variables. Instead of setting global environment variables, create a new file to store the environment variables and then pass the variables to the Concourse CI using the Systemd service.
Create a new environment file for Concourse web.
sudo nano /opt/concourse/web.env
Populate the file.
CONCOURSE_SESSION_SIGNING_KEY=/opt/concourse/session_signing_key
CONCOURSE_TSA_HOST_KEY=/opt/concourse/tsa_host_key
CONCOURSE_TSA_AUTHORIZED_KEYS=/opt/concourse/authorized_worker_keys
CONCOURSE_POSTGRES_USER=concourse
CONCOURSE_POSTGRES_PASSWORD=DBPassword
CONCOURSE_POSTGRES_DATABASE=concourse
CONCOURSE_BASIC_AUTH_USERNAME=admin
CONCOURSE_BASIC_AUTH_PASSWORD=StrongPass
CONCOURSE_EXTERNAL_URL=http://192.0.2.1:8080
Change the username and password of the BASIC_AUTH
if desired. Make sure that the path to the key files are correct and make sure that the correct value for username and password in the PostgreSQL database configuration is provided.
Similarly, create an environment file for the worker.
sudo nano /opt/concourse/worker.env
Populate the file.
CONCOURSE_WORK_DIR=/opt/concourse/worker
CONCOURSE_TSA_WORKER_PRIVATE_KEY=/opt/concourse/worker_key
CONCOURSE_TSA_PUBLIC_KEY=/opt/concourse/tsa_host_key.pub
CONCOURSE_TSA_HOST=127.0.0.1
As the environment files contain username and passwords, change its permissions so that it cannot be accessed by other users.
sudo chmod 600 /opt/concourse/*.env
Now create a new user for Concourse to run the web environment. This will ensure that the web server is running in an isolated environment.
sudo adduser --system concourse
Give the concourse user ownership over Concourse CI file’s directory.
sudo chown -R concourse:concourse /opt/concourse
Create a new systemd service file for the Concourse web service.
sudo nano /etc/systemd/system/concourse-web.service
Populate the file.
[Unit]
Description=Concourse CI web server
After=postgresql-9.6.service
[Service]
Type=simple
User=concourse
Group=concourse
Restart=on-failure
EnvironmentFile=/opt/concourse/web.env
ExecStart=/usr/bin/concourse web
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=concourse_web
[Install]
WantedBy=multi-user.target
Save and close the file. Create a new service file for the Concourse worker service.
sudo nano /etc/systemd/system/concourse-worker.service
Populate the file.
[Unit]
Description=Concourse CI worker process
After=concourse-web.service
[Service]
Type=simple
User=root
Group=root
Restart=on-failure
EnvironmentFile=/opt/concourse/worker.env
ExecStart=/usr/bin/concourse worker
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=concourse_worker
[Install]
WantedBy=multi-user.target
The web and worker service can now be started directly by running:
sudo systemctl start concourse-web concourse-worker
To enable the worker and web process to automatically start at boot time, run:
sudo systemctl enable concourse-worker concourse-web
To check the status of services, run:
sudo systemctl status concourse-worker concourse-web
If the service is not started, or in the FAILED
state, remove the cache from the /tmp
directory.
sudo rm -rf /tmp/*
Restart the services.
sudo systemctl restart concourse-worker concourse-web
Notice that this time the services have started correctly. The output upon verifying the status of the services should be simil.
[user@vultr ~]$ sudo systemctl status concourse-worker concourse-web
● concourse-worker.service - Concourse CI worker process
Loaded: loaded (/etc/systemd/system/concourse-worker.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2017-08-26 07:27:37 UTC; 55s ago
Main PID: 3037 (concourse)
CGroup: /system.slice/concourse-worker.service
└─3037 /usr/bin/concourse worker
Aug 26 07:27:42 vultr.guest concourse_worker[3037]: {"timestamp":"1503732462.934722900","source":"tsa","message":"t...""}}
Aug 26 07:27:42 vultr.guest concourse_worker[3037]: {"timestamp":"1503732462.941227913","source":"guardian","messag...0"}}
...
● concourse-web.service - Concourse CI web server
Loaded: loaded (/etc/systemd/system/concourse-web.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2017-08-26 07:27:37 UTC; 55s ago
Main PID: 3036 (concourse)
CGroup: /system.slice/concourse-web.service
└─3036 /usr/bin/concourse web
Aug 26 07:27:57 vultr.guest concourse_web[3036]: {"timestamp":"1503732477.925554752","source":"tsa","message":"tsa...ve"}}
Aug 26 07:28:02 vultr.guest concourse_web[3036]: {"timestamp":"1503732482.925430775","source":"tsa","message":"tsa...ve"}}
...
Hint: Some lines were ellipsized, use -l to show in full.
Adjust your firewall to allow port 8080, on which ATS is running and port 2222, on which TSA is running.
sudo firewall-cmd --zone=public --add-port=8080/tcp --permanent
sudo firewall-cmd --zone=public --add-port=2222/tcp --permanent
sudo firewall-cmd --reload
Connecting to the Server
Once the server is started, the web interface of the Concourse CI can be accessed by going to http://192.0.2.1:8080
in any browser. Log in using the username and password provided in the environment file.
To connect to the server using Fly, run:
fly -t my-ci login -c http://192.0.2.1:8080
The above command is used for initial login to the server. -t
is used to provide a target name. replace my-ci
with any desired target name. The above command will log in to the default team main
. It will ask for the username and password provided in the environment file.
The output will look like the following.
[user@vultr ~]$ fly -t my-ci login -c http://192.0.2.1:8080
logging in to team 'main'
username: admin
password:
target saved
The target login will be saved for a day. After that, it will expire.
To log out immediately.
fly -t my-ci logout
fly can be used to login to the server outside of the network, but only if the server has a public IP address and it is accessible from outside the network. The Windows or MacOS binary can be downloaded from the download site or from the web UI of the server.
Setting Up Nginx Reverse Proxy
Logins, and other information sent through the web UI to the Concourse server is not secured. The connection is not encrypted. An Nginx reverse proxy can be set up with a Let’s Encrypt free SSL.
Install the Nginx web server and Certbot, which is the client application for the Let’s Encrypt CA.
sudo yum -y install certbot-nginx nginx
Start and enable Nginx to automatically start at boot time:
sudo systemctl start nginx
sudo systemctl enable nginx
Before a request can be made for the certificates, port 80 and 443, or standard HTTP and HTTPS services, must be enabled through the firewall. Certbot will check the domain authority before issuing certificates.
sudo firewall-cmd --zone=public --add-service=http --permanent
sudo firewall-cmd --zone=public --add-service=https --permanent
Port 8080 no longer needs to be allowed through the firewall anymore because Concourse will now be run on the standard HTTPS port. Remove the firewall entry to allow port 8080.
sudo firewall-cmd --zone=public --remove-port=8080/tcp --permanent
sudo firewall-cmd --reload
Note
To obtain certificates from Let’s Encrypt CA, the domain for which the certificates are to be generated must be pointed towards the server. If not, make the necessary changes to the DNS records of the domain and wait for the DNS to propagate before making the certificate request again. Certbot checks the domain authority before providing the certificates.
Generate the SSL certificates.
sudo certbot certonly --webroot -w /usr/share/nginx/html -d ci.example.com
The generated certificates are likely to be stored in the /etc/letsencrypt/live/ci.example.com/
directory. The SSL certificate will be stored as fullchain.pem
and the private key will be stored as privkey.pem
.
Let’s Encrypt certificates expire in 90 days, so it is recommended auto renewal for the certificates is set up using cronjobs. Cron is a system service which is used to run periodic tasks.
Open the cron job file.
sudo crontab -e
Add the following line at the end of the file.
30 5 * * 1 /usr/bin/certbot renew --quiet
The above cron job will run every Monday at 5:30 AM. If the certificate is due for expiry, it will automatically be renewed.
Create a new virtual host.
sudo nano /etc/nginx/conf.d/concourse-ssl.conf
Populate the file.
server {
listen 80;
server_name ci.example.com;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name ci.example.com;
ssl_certificate /etc/letsencrypt/live/ci.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ci.example.com/privkey.pem;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/concourse.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://localhost:8080;
proxy_read_timeout 90;
proxy_redirect http://localhost:8080 https://ci.example.com;
}
}
Note: Replace ci.example.com
with the actual domain.
Edit the Environment file created for concourse Web.
sudo nano /opt/concourse/web.env
Change the value of CONCOURSE_EXTERNAL_URL
and also add two more lines at the end of the file.
CONCOURSE_EXTERNAL_URL=https://ci.example.com
CONCOURSE_BIND_IP=127.0.0.1
CONCOURSE_BIND_PORT=8080
Save the file and restart Concourse Web, Worker and Nginx web server:
sudo systemctl restart concourse-worker concourse-web nginx
All the data sent to and from the browser is now secured with SSL encryptions.
Want to contribute?
You could earn up to $300 by adding new articles
Suggest an update
Request an article