This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

United Manufacturing Hub

Learn how to contribute to the United Manufacturing Hub.

1 - Setup Local Environment

This document describes how to set up your local environment for contributing to the United Manufacturing Hub.

The following instructions describe how to set up your local environment for contributing to the United Manufacturing Hub.

You can use any text editor or IDE. However, we recommend using JetBrains GoLand.

Requirements

The following tools are required to contribute to the United Manufacturing Hub. Use the links to install the correct version for your operating system. We recommend using a package manager where possible (for Windows, we recommend using Chocolatey).

  • Git
  • Go version 1.19 or later
  • Docker version 20.10 or later
  • kubectl version 1.26 or later
  • Helm version 3.11 or later
  • k3d version 5.0 or later
  • GNU C Compiler version 12 or later. The gcc binaries must be in your PATH environment variable, and the go variable CGO_ENABLED must be set to 1. You can check this by running go env CGO_ENABLED in your terminal.

Other tools that are not required, but are recommended:

Fork the documentation repository

If you are not a member of the United Manufacturing Hub organization, you will need to fork the repository to your own GitHub account. This is done by clicking the Fork button in the top-right corner of the united-manufacturing-hub/united-manufacturing-hub repository page.

Clone the repository

Clone the repository to your local machine:

git clone https://github.com/<user>/united-manufacturing-hub.git
# or: git clone [email protected]:<user>/united-manufacturing-hub.git

Where <user> is your GitHub username, or united-manufacturing-hub if you are a member of the United Manufacturing Hub organization.

If you are not a member of the United Manufacturing Hub organization, you will need to add the upstream repository as a remote:

git remote add origin https://github.com/united-manufacturing-hub/united-manufacturing-hub.git
# or: git remote add upstream [email protected]:united-manufacturing-hub/united-manufacturing-hub.git

# Never push to upstream master
git remote set-url --push origin no_push

Install dependencies

Download the go dependencies:

make go-deps

Build the container images

These are the make targets to manage containers:

# Build the container images
make docker-build

# Push the container images
make docker-push

# Build and push the container images
make docker

You can pass the following variables to change the behavior of the make targets:

  • CTR_REPO: The container repository to push the images to. Defaults to ghcr.io/united-manufacturing-hub.
  • CTR_TAG: The tag to use for the container images. Defaults to latest.
  • CTR_IMG: Space-separated list of container images. Defaults to all the images in the deployment directory.

Run a cluster locally

To run a local cluster, run:

# Create a cluster that runs the latest version of the United Manufacturing Hub
make cluster-install

# Create a cluster that runs the local version of the United Manufacturing Hub
make cluster-install CHART=./deployment/helm/united-manufacturing-hub

You can pass the following variables to change the behavior of the make targets:

  • CLUSTER_NAME: The name of the cluster. Defaults to umh.
  • CHART: The Helm chart to use. Defaults to united-manufacturing-hub/united-manufacturing-hub.
  • VERSION: The version of the Helm chart to use. Default is empty, which means the latest version.
  • VALUES_FILE: The Helm values file to use. Default is empty, which means the default values.

Test

To run the unit tests, run:

make go-test-unit

To run e2e tests, run:

make helm-test-upgrade

# To run the upgrade test with data
make helm-test-upgrade-with-data

Other useful commands

# Display the help for the Makefile
make help

# Pass the PRINT_HELP=y flag to make to print the help for each target
make cluster-install PRINT_HELP=y

What’s next

2 - Coding Conventions

This document outlines a collection of guidelines, style suggestions, and tips for writing code in the different programming languages used throughout the Kubernetes project.

Code conventions

  • Bash

  • Go

    • Go Code Review Comments
    • Effective Go
    • Know and avoid Go landmines
    • Comment your code.
    • Command-line flags should use dashes, not underscores
    • Naming
      • Please consider package name when selecting an interface name, and avoid redundancy. For example, storage.Interface is better than storage.StorageInterface.
      • Do not use uppercase characters, underscores, or dashes in package names.
      • Please consider parent directory name when choosing a package name. For example, pkg/controllers/autoscaler/foo.go should say package autoscaler not package autoscalercontroller.
        • Unless there’s a good reason, the package foo line should match the name of the directory in which the .go file exists.
        • Importers can use a different name if they need to disambiguate.
      • Locks should be called lock and should never be embedded (always lock sync.Mutex). When multiple locks are present, give each lock a distinct name following Go conventions: stateLock, mapLock etc.

Testing conventions

  • All new packages and most new significant functionality must come with unit tests.
  • Significant features should come with integration and/or end-to-end.
  • Do not expect an asynchronous thing to happen immediately—do not wait for one second and expect a pod to be running. Wait and retry instead.

Directory and file conventions

  • Avoid package sprawl. Find an appropriate subdirectory for new packages.
    • Libraries with no appropriate home belong in new package subdirectories of pkg/util.
  • Avoid general utility packages. Packages called “util” are suspect. Instead, derive a name that describes your desired function. For example, the utility functions dealing with waiting for operations are in the wait package and include functionality like Poll. The full name is wait.Poll.
  • All filenames should be lowercase.
  • Go source files and directories use underscores, not dashes.
    • Package directories should generally avoid using separators as much as possible. When package names are multiple words, they usually should be in nested subdirectories.
  • Document directories and filenames should use dashes rather than underscores.
  • Go code for normal third-party dependencies is managed using go modules.

3 - Automation Tools

This section contains the description of the automation tools used in the United Manufacturing Hub project.

Automation tools are an essential part of the United Manufacturing Hub project. They automate the building and testing of the project’s code, ensuring that it remains of high quality and stays reliable.

We rely on GitHub Actions for running the pipelines, which are defined in the .github/workflows directory of the project’s repository.

Here’s a brief overview of each workflow:

Build Docker Images

This pipeline builds and pushes all the Docker images for the project, tagging them using the branch name or the git tag. This way there is always a tagged version for the latest release of the UMH, as well as specific version for each branch to use for testing.

It runs on push events only when relevant files have been changed, such as the Dockerfiles or the source code.

Codecov

This pipeline executes go test and uploads the results to codecov, to make sure that unit tests are passing and they cover most of the code.

It runs on both push and pull request events.

GitGuardian Scan

This pipeline scans the code for security vulnerabilities, such as exposed secrets.

It runs on both push and pull request events.

Helm Checks

This pipeline runs checks on Helm configuration. It executes helm lint for the default values.yaml and for any custom values used for testing. Additionally, runs checkov to prevent misconfigurations and vulnerabilities, along with best practices checks.

It runs on push events only when Helm template files have been changed.

Test Deployment

This pipeline group verifies that the current changes can be successfully installed and that data flows correctly. There are two pipelines: a “tiny” version with the minimum amount of services needed to run the stack, and a “full” version with as many services as possible.

Each pipeline has two jobs. The first job installs the stacks with the current changes, and the second job tries to upgrade from the latest stable version to the current changes.

A test is run in each workflow to verify that simulated data flows through MQTT, NodeRed, Kafka, and TimescaleDB. In the full version, an additional test for sensorconnect is run, using a mocked sensor to verify the data flow.

It runs on pull request events when the Helm configuration or the source code changes.

4 - Release Process

This page describes how to release a new version of the United Manufacturing Hub.

Releases are coordinated by the United Manufacturing Hub team. All the features and bug fixes due for a release are tracked in the internal project board.

Once all the features and bug fixes for a release are ready and merged into the main branch, the release process can start.

Prerelease

The prerelease process is used to test the release before it is published. If bugs are found during the prerelease, they can be fixed and the release process can be restarted. Once the prerelease is finished, the release can be published.

  1. Create a prerelease branch from main:

    git checkout main
    git pull
    git checkout -b <next-version>-prerelease1
    
  2. Update the version and appVersion fields in the Chart.yaml file to the next version:

    version: <next-version>-prerelease1
    appVersion: <next-version>-prerelease1
    
  3. Navigate to the deployment/helm-repo directory and run the following commands:

    helm package ../united-manufacturing-hub
    helm repo index --url https://v<next-version>-prerelease1.united-manufacturing-hub.pages.dev --merge index.yaml .
    

    Pay attantion to use - instead of . as a separator in <next-version>.

  4. Commit and push the changes:

    git add .
    git commit -m "build: <next-version>-prerelease1"
    git push origin <next-version>-prerelease1
    

Test

All the new releases must be thoroughly tested before they can be published. This includes specific tests for the new features and bug fixes, as well as general tests for the whole stack.

General tests include, but are not limited to:

  • Deploy the stack with flatcar
  • Upgrade the stack from the previous version
  • Deploy the stack on Karbon 300 and test with real sensors

If any bugs are found during the testing phase, they must be fixed and pushed to the prerelease branch. Multiple prerelease versions can be created if necessary.

Release

Once all the tests have passed, the release can be published. Merge the prerelease branch into main and create a new release branch.

  1. Create a release branch from main:

    git checkout main
    git pull
    git checkout -b <next-version>
    
  2. Update the version and appVersion fields in the Chart.yaml file to the next version:

    version: <next-version>
    appVersion: <next-version>
    
  3. Navigate to the deployment/helm-repo directory and run the following commands:

    helm package ../united-manufacturing-hub
    helm repo index --url https://repo.umh.app --merge index.yaml .
    
  4. Commit and push the changes, tagging the release:

    git add .
    git commit -m "build: <next-version>"
    git tag <next-version>
    git push origin <next-version> --tags
    
  5. Merge the release branch into main and create a new release from the tag on GitHub.