Introduction

This documents how to use the RAIL Container Platform.

RAIL allows product teams at UiB to host container based applications on a shared infrastructure operated by the ITA Plattform Team.

RAIL provides shared Kubernetes clusters running on top of the NREC infrastructure. Each product team get access to and controls one or more namespaces that are present in all RAIL clusters.

RAIL currently provide 4 distinct clusters based on available regions and classified by environment type. These are:

• bgo1-prod

• bgo1-test

• osl1-prod

• osl1-test

and this is a overview diagram of the service and the teams involved:

The regions are Bergen (BGO) and Oslo (OSL). Production clusters are where the real production loads are running, test clusters are for everything else. If we have outages, then production clusters will be prioritized and it’s easier to restore them if they are not filled with all kind of test workloads as well. Test clusters are for experiments and test instances of applications. The compute and storage resources as well as restrictions might differ between the types of environments.

RAIL is based on a flavor and distribution of Kubernetes called Welkin from the company Elastisys. This distribution was formerly known as Compliant Kubernetes.

The Welkin logo

Get started

To get started deploying applications as a product team you need to request a RAIL namespace. This is a manual process where you send request to the Platform Team through UiBhjelp. The main namespace for a team will get the symbolic name/acronym that match the official organization unit corresponding to the team. The people that get access can be the full team or a limited subset. Each person that should have access also need to provide the platform team with their SSH public key.

RAIL does not expose the Kubernetes API directly to the Internet, which means you can’t easily use clients like kubectl directly from your own workstation. Each cluster have a login host that you need to ssh into and run kubectl from. Authentication to the login host is based on the SSH key provided during the setup process.

The DNS name of the login host is login-01.<cluster-name>.rail.uhdc.no where you replace the dash in the cluster name with period. So to log into the bgo1-test cluster’s login host run this command:

ssh login-01.bgo1.test.rail.uhdc.no

The first time you log in the system is going to install stuff in your home directory to set up communication with the Kubernetes API. This communication is protected by authenticating you using your UiB account (using OIDC against Microsoft login service) and will require a round trip to your browser which will require extra port forwarding setup with the ssh command.

The automatic setup gives you a dummy default namespace that’s not functional. You will have to manually adjust the contexts.context.namespace setting of the ~/.kube/config file. Running this command should do the job:

kubectl config set-context --current --namespace=adm-it-xxx

If you ever want to revert this configuration file to the RAIL default, just remove it, log in and set your default namespace again.

To avoid conflicts when forwarding ports via SSH, each user is provided with individual ports. To run kubectl on the bgo1-test cluster run a command like this from your work station:

RAIL_PORT=$(ssh login-01.bgo1.test.rail.uhdc.no grep -- --listen-address=127.0.0.1: .kube/config | cut -d: -f2)
ssh -L$RAIL_PORT:localhost:$RAIL_PORT login-01.bgo1.test.rail.uhdc.no kubectl get all

The -L$RAIL_PORT:localhost:$RAIL_PORT part is required for the first invocation and when reauthentication is requested to allow kubectl to login the user though the web-browser on your local machine. It might be a good idea to make an command line alias out of this.

or, alternatively, you can create an entry like this in your ~/.ssh/config file:

Host rail-bgo1-test
    HostName login-01.bgo1.test.rail.uhdc.no
    User <your_username>
    LocalForward RAIL_PORT localhost:RAIL_PORT

Replace RAIL_PORT with the variable value calculated above since the ~/.ssh/config configuration files does not support variable expansion. Note that your RAIL_PORT will be different for each cluster.

Now you should be ready to control and monitor your namespaces using kubectl. You might for instance try to run a job or to deploy a service. The login host also have the handy k9s monitoring tool available.

When you have completed logging into ALL login-01.<cluster-name>.rail.uhdc.no servers you have the option to download (git pull) https://itgit.app.uib.no/it/sv/ark/ansible-rail and install via ansible some nify helper scripts, aliases and cli-prompt (PROMPT_COMMAND).

GitOps

When you request a team namespace you are also allocated a deployment Git repository at gitops that is linked to this namespace. The kubernetes resource files maintained in the main branch of this repository is automatically synced with the namespace. This should be the primary way that teams control their deployments to RAIL. When you want something to change you commit and push a change to the deployment repository and it will automatically be reflected the code that runs on RAIL. This style of working is known as GitOps and is considered best practice.

See the GitOps documentation for more details.

Summary of services provided by RAIL

• Kubernetes API to manage workloads, services, secrets, etc.

• GitOps workflow

• Linux Container runtime on x86_64 architecture with up to 4GB of virtual memory

• Persistent storage volumes up to 100GB with backup

• Services exposed on the Internet through HTTPS.

• SSL certificate management for cluster specific DNS-subdomains.

• E-mail forwarding

Restrictions in RAIL

There are some important restrictions that RAIL imposes to make the platform secure and which might make it harder to make your containers run on this platform:

• containers are not allowed to run as root user (uid=0).

• containers cannot bind to ports below 1024 (as only root is allowed to do that).

• use of the image tag :latest is forbidden (also in the implied form)

• containers can only pull images from approved registries.

• containers must declare their resource limits.

Other services

This diagram shows some of the other services that are part of the RAIL platform. These services will be available under the indicated domain name for each cluster.

Links to services in each cluster

Description\Cluster	bgo1-prod	bgo1-test	osl1-prod	osl1-test
Graphs and metrics on how the clusters are doing	grafana	grafana	grafana	grafana
Visibility into logs	opensearch	opensearch	opensearch	opensearch