SF-Operator: a Zuul-based CI infrastructure for OpenShift

SF-Operator is a Kubernetes deployment system to install Software Factory (SF) on the OpenShift Container Platform. SF is a Continuous Integration (CI) service based on Zuul to provide project gating with Ansible for developer platforms like GitLab using cloud providers like OpenStack.

Getting Started

To try SF-Operator you will need:

• A Linux system to run the operator standalone (RHEL, CentOS or Fedora are supported).
• A copy of the source code.
• Access to a Kubernetes or OpenShift cluster (recommended).

Run the following commands:

git clone https://softwarefactory-project.io/r/software-factory/sf-operator
cd sf-operator
./tools/deploy.sh

This procedure is expected to work out of the box on a fresh system, please create a bug report if it ever fails.

You now have the Software Factory services running at the default https://sfop.me domain. The next steps are:

• Configure the custom resource (CR) to change the FQDN, add Zuul connections and Nodepool providers.
• Configure public access to the SF services using a Route or Ingress.
• Create a Zuul tenant in the provided project config.
• Manage the project config through the config-update pipelines by hosting the configuration on your developer platform.

The next sections introduces the SF architecture and the installation options.

Architecture Overview

Software Factory is composed of the following services:

• Zuul, for running the CI.
• Gateway, for the HTTP frontend.
• Nodepool, for the resource providers.
• LogServer, for the build logs.
• HoundSearch, for code search.
• LogJuicer, for build analysis.
• Weeder, for inspecting whole tenant config.

Internally, SF leverage the following services:

• ZooKeeper, for the Zuul state.
• MariaDB, for storing the build results.
• GitServer, for hosting the internal tenant config.
• Gerrit, for the initial config project location until it is moved to an external location.

SF-Operator deployment is defined with the following elements:

• The Software Factory custom resource (CR) to manage the services configuration.
• The project config repository to manage the Zuul tenants and Nodepool providers.

Install

SF-Operator supports OpenShift and Kubernetes, and it is tested with the following local cluster deployment:

• MicroShift for OpenShift
• Minikube for Kubernetes

Beside the Getting Started process, here are the available installation modes:

• Run the SF-Operator in standalone mode (recommended).
• Install the CRD and deploy the SF-Operator on the cluster to manage the SF resources with kubectl.
• Install the OLM catalog to manage the SF-Operator installation through the web console.

For example, here is a standard configuration resource (CR), adapted from the getting started one, which is available in sf.yaml:

apiVersion: sf.softwarefactory-project.io/v1
kind: SoftwareFactory
metadata:
  name: my-sf
spec:
  fqdn: "sfop.me"
  config-location:
    name: sf/project-config
    zuul-connection-name: my-gitlab
  zuul:
    gitlabconns:
      - name: my-gitlab
        server: gitlab.me
        baseurl: https://gitlab.me
        secrets: gitlab-secret

Which can be installed by running:

# Make sure the ~/.kube/config is valid
oc login --token=SECRET --server=https://openshift.me:6443

# Apply the CR
go run ./main.go deploy ./my-sf.yaml

Checkout the other documentations:

• Deploying Zuul and dependencies with SF-Operator
• Installing the Operator (OLM)
• Developing the SF-Operator

About

SF-Operator is the natural evolution of the Software Factory project: the 3.8.x release of Software Factory saw the containerization of every major service, but was still delivered as RPM packages, in the form of a custom CentOS 7 distribution. SF-Operator builds upon this containerization effort to move from a distro-centric approach to a cloud-native deployment. This is also an opportunity to focus on the leanest service suite needed to provide a working gated CI infrastructure; hence a scope reduced to Zuul and its dependencies only.

SF-Operator is built mostly in Go upon the Operator Framework, with the aim of reaching the highest capability level that can be achieved, in order to bring a modern, scalable gated CI alternative to OpenShift users, with the least friction from operation as possible.

Furthermore, SF-Operator takes advantage of some of the specificities of OpenShift as a container orchestration platform:

• Improved ingress management with OpenShift's Route Custom Resources
• Integration with OLM for streamlined operator and operands' lifecycle management
• If enabled in OpenShift, SF-Operator comes with default monitoring and alerting configurations that can be used out of the box. The default alerting rules are honed from years of maintaining and running several large Zuul deployments at scale for Fedora, Ansible and RDO.
• If enabled, OpenShift provides application logs aggregation with its logging subsystem out of the box.

Finally, we also provide a Command Line Interface (CLI) to simplify common tasks related to the operator, management of the operands, development and testing.

Status

The current project status is: Beta

Capability Levels

• Level 1 - Basic Install - 10/10
  • Zuul Scheduler: ✅
  • Zuul Executor: ✅
  • Zuul Web: ✅
  • Zuul Merger: ✅
  • Nodepool Launcher: ✅
  • Nodepool Builder: ✅
  • Zookeeper: ✅
  • MariaDB: ✅
  • Log Server: ✅
  • Internal Config Repository, bootstrapped pipelines and default jobs: ✅
• Level 2 - Seamless upgrades - 2/2
  • Operator: ✅
  • Operands: ✅
• Level 3 - Full Lifecycle - 3/5
  • SF 3.8.x migration ❌
  • Backup: ✅
  • Restore: ✅
  • Rolling deployments: ❌
  • Reconfiguration: ✅
• Level 4 - Deep Insights - 1/3
  • Operator metrics: ❌
  • Operand metrics: ✅
  • Alerts: ❌ (WIP)
• Level 5 - Auto pilot - 0/3
  • Auto-scaling : ❌
  • Auto-healing: ❌
  • Auto-tuning: ❌

Getting Started

• Installing the Operator
• Deploying Zuul and dependencies with SF-Operator

Documentation

• Operator documentation: for OpenShift cluster administrators, this documentation covers installing SF-Operator and managing the operator's lifecycle.
• Deployment documentation: this documentation covers the essentials for people or teams who intend to deploy and manage Zuul and its dependencies through the SF-Operator.
• Developer documentation: this documentation describes how to set up a development and testing environment to develop the SF-Operator.
• End User documentation: for users of a Software Factory instance. This documentation mostly describes the Software Factory's config repository usage (configuration-as-code).
• CLI refererence

Getting Help

Should you have any questions or feedback concerning the SF-Operator, you can:

• Join our Matrix channel
• Send an email to [email protected]
• File an issue for bugs and feature suggestions

Contributing

Refer to CONTRIBUTING.md.

Licence

Sf-operator is distributed under the Apache License.