Skip to main content

Configure a Kubernetes build farm to use self-signed certificates

CI build infrastructure pods can interact with servers using self-signed certificates. This option is useful for organizations that prefer to use internal certificates instead of certificates generated by a public Certificate Authority (CA).

info
  • This topic assumes that you are familiar with how to implement SSL in Kubernetes. General information about implementing SSL is outside the scope of this topic.
  • With a Kubernetes cluster build infrastructure, all Build and Push steps use kaniko. Kaniko uses the path /kaniko/ssl/certs/additional-ca-cert-bundle.crt to read certificates.
  • Harness uses a UBI image for the Git Clone step. UBI reads certificates from /etc/ssl/certs/ca-bundle.crt.
  • Different base images use different paths as their default certificate location. For example, Alpine images use this path to recognize certificates: /etc/ssl/certs/ca-certificates.crt For any other image, make sure you verify the default certificate path.

Enable self-signed certificates​

  1. Create a Kubernetes secret or config map with the required certificates in the same namespace used by the Harness delegate. For example:

    apiVersion: v1  
    kind: Secret  
    metadata:  
      name: addcerts  
      namespace: harness-delegate-ng  
    type: Opaque  
    stringData:                             
      ca.bundle: |  
        -----BEGIN CERTIFICATE-----  
        XXXXXXXXXXXXXXXXXXXXXXXXXXX  
        -----END CERTIFICATE-------  
        -----BEGIN CERTIFICATE-----  
        XXXXXXXXXXXXXXXXXXXXXXXXXXX  
        -----END CERTIFICATE-------

  2. Mount the secret as a volume on the delegate pod.

    For instructions, go to the Kubernetes documentation on Configuring a Pod to Use a Volume for Storage.

    In the delegate pod, you must specify DESTINATION_CA_PATH. Provide a comma-separated list of paths in the build pod where you want the certs to be mounted, and mount your certificate files to opt/harness-delegate/ca-bundle.

           env:
            - name: DESTINATION_CA_PATH
              value: "/etc/ssl/certs/ca-bundle.crt,/kaniko/ssl/certs/additional-ca-cert-bundle.crt"
              volumeMounts:
               - name: certvol
                 mountPath: /opt/harness-delegate/ca-bundle/ca.bundle
                 subPath:  ca.bundle
           volumes:
            - name: certvol
              secret:
                secretName: addcerts
                items:
                 - key: ca.bundle
                   path: ca.bundle

    Both CI build pods and the SCM client on the delegate support this method.

    warning

    Make sure the destination path is not same as the default CA certificate path of the corresponding container image.

    If you want to override the default certificate file, make sure the Kubernetes secret or config map (from step one) includes all certificates required by the pipelines that will use this build infrastructure.

Legacy: CI_MOUNT_VOLUMES

Prior to the introduction of DESTINATION_CA_PATH, you used ADDITIONAL_CERTS_PATH and CI_MOUNT_VOLUMES to mount certs.

The legacy method is still supported, but Harness recommends DESTINATION_CA_PATH. If you include both, DESTINATION_CA_PATH takes precedence. If Harness can't resolve DESTINATION_CA_PATH, it falls back to CI_MOUNT_VOLUMES and ADDITIONAL_CERTS_PATH.

You must specify both ADDITIONAL_CERTS_PATH and CI_MOUNT_VOLUMES.

For ADDITIONAL_CERTS_PATH, provide the path to the certificates in the delegate, such as /tmp/ca.bundle.

For CI_MOUNT_VOLUMES, provide a comma-separated list of source:destination mappings where source is the certificate path on the delegate, and destination is the path where you want to expose the certificates on the build containers. For example:

/tmp/ca.bundle:/etc/ssl/certs/ca-bundle.crt,/tmp/ca.bundle:/kaniko/ssl/certs/additional-ca-cert-bundle.crt

The CI_MOUNT_VOLUMES list must include all certificates that your build containers need to interact with external services.

       env:  
        - name: ADDITIONAL_CERTS_PATH  
          value: /tmp/ca.bundle  
        - name: CI_MOUNT_VOLUMES  
          value: /tmp/ca.bundle:/etc/ssl/certs/ca-bundle.crt,/tmp/ca.bundle:/kaniko/ssl/certs/additional-ca-cert-bundle.crt  
          volumeMounts:  
           - name: certvol  
             mountPath: /tmp/ca.bundle  
             subPath:  ca.bundle 
       volumes:  
        - name: certvol  
          secret:  
            secretName: addcerts  
            items:  
             - key: ca.bundle  
               path: ca.bundle
  1. Restart the delegate. Once it is up and running, exec into the container and ensure that the volume exists at the mounted path and contains your certificates.

Additional configuration for pipelines with STO scan steps​

If you have STO scan steps in your pipeline, do the following:

  1. Follow the steps to enable self-signed certificates.

  2. If you're storing your certificates in a local registry and need to run Docker-in-Docker, specify the local certificate path on the delegate.

    Suppose your self-signed certs are stored at https://my-registry.local.org:799 and you log in like this:

    docker login my-registry.local.org:799

    You would then need to configure or extend the DESTINATION_CA_PATH on the delegate as follows. Note the value path:

    env:
    - name: DESTINATION_CA_PATH
               value: "/etc/docker/certs.d/my-registry.local.org:799/ca.crt"
             volumeMounts:
             - name: certvol
               mountPath: opt/harness-delegate/ca-bundle
               subPath:  ca.bundle
           volumes:
           - name: certvol
             secret:
               secretName: addcerts
               items:
               - key: ca.bundle
                 path: ca.bundle

  3. Complete the additional steps and requirements described in Add custom certificates to a delegate.

Troubleshoot Kubernetes cluster build infrastructures​

Go to the CI Knowledge Base for questions and issues related to Kubernetes cluster build infrastructures, including use of self-signed certificates, such as: