This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Production Guide

This section contains information about how to use the stack in a production environment.

1: Installation

1.1: Flatcar Installation (Bare Metal)
1.2: Flatcar Installation (Virtual Machine)
1.3: Local k3d Installation

2: Upgrading

2.1: Upgrade to v0.9.14
2.2: Upgrade to v0.9.13
2.3: Upgrade to v0.9.12
2.4: Upgrade to v0.9.11
2.5: Upgrade to v0.9.10
2.6: Upgrade to v0.9.9
2.7: Upgrade to v0.9.8
2.8: Upgrade to v0.9.7
2.9: Upgrade to v0.9.6
2.10: Upgrade to v0.9.5
2.11: Upgrade to v0.9.4

3: Administration

3.1: Access the Database
3.2: Access Services From Within the Cluster
3.3: Access Services Outside the Cluster
3.4: Access Kafka Outside the Cluster
3.5: Expose Grafana to the Internet
3.6: Install Custom Drivers in NodeRed
3.7: Execute Kafka Shell Scripts
3.8: Reduce database size
3.9: Delete Assets from the Database
3.10: Change the Language in Factoryinsight
3.11: Explore Cached Data
3.12: Optimize Time Consuming Queries
3.13: Optimize Database Datatypes

4: Backup & Recovery

4.1: Backup and Restore the United Manufacturing Hub
4.2: Backup and Restore Database
4.3: Import and Export Node-RED Flows

5: Security

5.1: Change VerneMQ ACL Configuration
5.2: Enable RBAC for the MQTT Broker
5.3: Setup PKI for the MQTT Broker

1 - Installation

This section contains guides on how to install the United Manufacturing Hub.

Learn how to install the United Manufacturing Hub using completely Free and Open Source Software.

1.1 - Flatcar Installation (Bare Metal)

This page describes how to deploy the United Manufacturing Hub on Flatcar Linux on bare metal.

Here is a step-by-step guide on how to deploy the UMH stack on Flatcar Linux, a Linux distribution designed for container workloads, with high security and low maintenance.

This is a good option if you want to deploy the UMH stack on edge devices or IPCs.

Before you begin

Your system must meet the following requirements before you can install the United Manufacturing Hub:

CPU cores: 4
Memory size: 8 GB
Hard disk size: 32 GB

You need the latest version of our iPXE boot image:

ipxe-x86_64-efi: for reasonably modern systems.
ipxe-x86_64-bios: for older systems.
ipxe-arm64-efi: for ARM systems. Currently, Raspberry Pi 4 is not supported.

The image needs to be written to a USB stick. If you want to know how to do this, follow our guide on how to flash an operating system onto a USB-stick.

You also need a computer with an SSH client (most modern operating systems already have it) and either UMHLens or OpenLens installed.

Additionally, this guide assumes a configuration similar to the following:

%%{ init: { 'flowchart': { 'curve': 'bumpY' } } }%% flowchart LR A(Internet) -. WAN .- B[Router] subgraph Internal network B -- LAN --- C[Edge device] B -- LAN --- D[Your computer] end

For optimal functionality, we recommend assigning a static IP address to your edge device. This can be accomplished through a static lease in the DHCP server or by setting the IP address during installation. Changing the IP address of the edge device after installation may result in certificate issues, so we strongly advise against doing so. By assigning a static IP address, you can ensure a more stable and reliable connection for your edge device.

Install Flatcar Linux on the edge device

Connect the USB stick to the edge device and boot it. Each device has a different way of booting from USB, so you need to consult the documentation of your device.
Accept the License.
Select the correct network settings. If you are unsure, select DHCP, but keep in mind that a static IP address is strongly recommended.
Select the correct drive to install Flatcar Linux on. If you are unsure, check the troubleshooting section.
Check that the installation settings are correct and press Confirm to start the installation.

Now the installation will start. You should see a green command line soon after, that says [email protected] ~$~. Now remove the USB stick from the device. At this point the system is still installing. After a few minutes, depending on the speed of your network, the installation will finish and the system will reboot. Now you should see a grey login prompt that says flatcar-1-umh login:, as well as the IP address of the device.

Please note that the installation may take some time. This largely depends on the available resources including network speed and system performance.

Connect to the edge device

Now you can leave the edge device and connect to it from your computer via SSH.

If you are on Windows 11, we recommend using the default Windows terminal, that you can find by typing terminal in the Windows search bar or Start menu. Next, connect to the edge device via SSH, using the IP address you saw on the login prompt:

ssh [email protected]<ip-address>

If you are not on Windows 11, you can use MobaXTerm to connect to the edge device via SSH. Open MobaXTerm and click on Session in the top left corner. Then click on SSH and enter the IP address of the edge device in the Remote host field. Click on Advanced SSH settings and enter core in the Username field. Click on Save and then on Open.

The default password for the core user is umh.

Import the cluster configuration

From your SSH session, run the following command to get the cluster configuration:

sudo kubectl config view --raw

The output should look similar to this:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://127.0.0.1:6443
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    client-certificate-data: <long string>
    client-key-data: <long string>

Copy the output.
Open UMHLens / OpenLens, click on the three horizontal lines in the upper left corner and choose Files > Add Cluster.
Paste the output.

Update the server field to the IP address of the device, e.g.:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://192.168.0.123:6443 # <- update this
...

If you want, you can also update the name field to something more meaningful, e.g.:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://192.168.0.123:6443
    name: my-edge-device # <- update this
...

Click on Add clusters.

Access the UMH stack

Open UMHLens / OpenLens on your device.
From the homepage, click on Browse Clusters in Catalog. You should see all your clusters.
Click on a cluster to connect to it.
Navigate to Helm > Releases and change the namespace from default to united-manufacturing-hub in the upper right corner.
Select the united-manufacturing-hub Release to inspect the release details, the installed resources, and the Helm values.

Troubleshooting

To check the status of the installation, run the following command:

systemctl status installer

If the installation is still running, you should see something like this:

● installer.service - Flatcar Linux Installer
     Loaded: loaded (/usr/lib/systemd/system/installer.service; static; vendor preset: enabled)
     Active: active (running) since Wed 2021-05-12 14:00:00 UTC; 1min 30s ago

Otherwise, the installation failed. You can check the logs to see what went wrong.

I don’t know which drive to select

You can check the drive type from the manual of your device.

For SATA drives (spinning hard disk or SSD), the drive type is SDA.
For NVMe drives, the drive type is NVMe.

If you are unsure, you can boot into the edge device with any Linux distribution and run the following command:

lsblk

The output should look similar to this:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 223.6G  0 disk
├─sda1   8:1    0   512M  0 part /boot
└─sda2   8:2    0 223.1G  0 part /
sdb      8:0    0  31.8G  0 disk
└─sdb1   8:1    0  31.8G  0 part /mnt/usb

In this case, the drive type is SDA. Generally, the drive type is the name of the first drive in the list, or at least the drive that doesn’t match the size of the USB stick.

I can access the cluster but there are no resources

First completely shut down UMHLens / OpenLens (from the system tray). Then start it again and try to access the cluster.

If that doesn’t work, access the edge device via SSH and run the following command:

systemctl status k3s

If the output contains a status different from active (running), the cluster is not running. Otherwise, the UMH installation failed. You can check the logs with the following commands:

systemctl status umh-install
systemctl status helm-install

If any of the commands returns some errors, is probably easier to reinstall the system.

What’s next

You can follow the Getting Started guide to get familiar with the UMH stack.
If you already know your way around the United Manufacturing Hub, you can follow the Administration guides to configure the stack for production.

1.2 - Flatcar Installation (Virtual Machine)

This page describes how to deploy the United Manufacturing Hub on Flatcar Linux in a virtual machine.

Here is a step-by-step guide on how to deploy the UMH stack on Flatcar Linux, a Linux distribution designed for container workloads, with high security and low maintenance, in a virtual machine.

This is a good option if you want to deploy the UMH stack on a virtual machine to try out the installation process or to test the UMH stack.

Before you begin

You need the latest version of our iPXE boot image.

The image needs to be written to a USB stick. If you want to know how to do this, follow our guide on how to flash an operating system onto a USB-stick.

You also need to have a virtual machine software installed on your computer. We recommend VirtualBox, which is free and open source, but other solutions are also possible.

Additionally, you need to have either UMHLens or OpenLens installed.

Create a virtual machine

Create a new virtual machine in your virtual machine software. Make sure to use the following settings:

Operating System: Linux
Version: Other Linux (64-bit)
CPU cores: 4
Memory size: 8 GB
Hard disk size: 32 GB

Also, the network settings of the virtual machine must allow communication with the internet and the host machine. If you are using VirtualBox, you can check the network settings by clicking on the virtual machine in the VirtualBox manager and then on Settings. In the Network tab, make sure that the Adapter 1 is set to Bridged Adapter.

Install Flatcar Linux

Start the virtual machine.
Accept the License.
Set a static IP address.
Select sda as the disk.
Select Confirm.

Now the installation will start. You should see a green command line soon after, that says [email protected] ~$~. At this point the system is still installing. After a few minutes, depending on the speed of your network, the installation will finish and the system will reboot.

By default, it will reboot into the installation environment. Just shut down the virtual machine and remove the ISO image from the CD drive, then boot the virtual machine again. This way, the installation process will continue, at the end of which you will a grey login prompt that says flatcar-1-umh login:, as well as the IP address of the device.

Please note that the installation may take some time. This largely depends on the available resources including network speed and system performance.

Connect to the virtual machine

You can leave the virtual machine running and connect to it using SSH, so that is easier to work with it.

Open a terminal on your computer and connect to the edge device via SSH, using the IP address you saw on the login prompt:

ssh [email protected]<ip-address>

If you are on Windows, you can use MobaXTerm to connect to the edge device via SSH. Open MobaXTerm and click on Session in the top left corner. Then click on SSH and enter the IP address of the edge device in the Remote host field. Click on Advanced SSH settings and enter core in the Username field. Click on Save and then on Open.

The default password for the core user is umh.

Import the cluster configuration

From your SSH session, run the following command to get the cluster configuration:

sudo kubectl config view --raw

The output should look similar to this:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://127.0.0.1:6443
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    client-certificate-data: <long string>
    client-key-data: <long string>

Copy the output.
Open UMHLens / OpenLens, click on the three horizontal lines in the upper left corner and choose Files > Add Cluster.
Paste the output.

Update the server field to the IP address of the device, e.g.:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://192.168.0.123:6443 # <- update this
...

If you want, you can also update the name field to something more meaningful, e.g.:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <long string>
    server: https://192.168.0.123:6443
    name: my-edge-device # <- update this
...

Click on Add clusters.

Access the UMH stack

Open UMHLens / OpenLens on your device.
From the homepage, click on Browse Clusters in Catalog. You should see all your clusters.
Click on a cluster to connect to it.
Navigate to Helm > Releases and change the namespace from default to united-manufacturing-hub in the upper right corner.
Select the united-manufacturing-hub Release to inspect the release details, the installed resources, and the Helm values.

Troubleshooting

To check the status of the installation, run the following command:

systemctl status installer

If the installation is still running, you should see something like this:

● installer.service - Flatcar Linux Installer
     Loaded: loaded (/usr/lib/systemd/system/installer.service; static; vendor preset: enabled)
     Active: active (running) since Wed 2021-05-12 14:00:00 UTC; 1min 30s ago

Otherwise, the installation failed. You can check the logs to see what went wrong.

I can access the cluster but there are no resources

First completely shut down UMHLens / OpenLens (from the system tray). Then start it again and try to access the cluster.

If that doesn’t work, access the virtual machine via SSH and run the following command:

systemctl status k3s

If the output contains a status different from active (running), the cluster is not running. Otherwise, the UMH installation failed. You can check the logs with the following commands:

systemctl status umh-install
systemctl status helm-install

If any of the commands returns some errors, is probably easier to reinstall the system.

I can’t SSH into the virtual machine

If you can’t SSH into the virtual machine, make sure that the network settings for the virtual machine are correct. If you are using VirtualBox, you can check the network settings by clicking on the virtual machine in the VirtualBox manager and then on Settings. In the Network tab, make sure that the Adapter 1 is set to NAT.

Disable any VPNs that you might be using.

What’s next

You can follow the Getting Started guide to get familiar with the UMH stack.
If you already know your way around the United Manufacturing Hub, you can follow the Administration guides to configure the stack for production.

1.3 - Local k3d Installation

This page describes how to deploy the United Manufacturing Hub locally using k3d.

This can now be done using the Community Edition of the Management Console. Go check it out!

Here is a step-by-step guide on how to deploy the UMH stack using k3d, a lightweight wrapper to run k3s in Docker. k3d makes it very easy to create single- and multi-node k3s clusters in Docker, e.g. for local development on Kubernetes.

Before you begin

Your system must meet the following requirements before you can install the United Manufacturing Hub:

CPU cores: 4
Memory size: 8 GB
Hard disk size: 32 GB

You also need to have Docker up and running and either UMHLens or OpenLens installed.

Install dependencies

Install kubectl. Refer to the kubectl installation if you need help.

choco install kubernetes-cli

brew install kubectl

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Install Helm. Refer to the Helm installation if you need help.

choco install kubernetes-helm

brew install helm

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Install k3d. Refer to the k3d installation if you need help.

choco install k3d

brew install k3d

curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

Create a cluster

Create a cluster.
```
k3d cluster create united-manufacturing-hub --api-port 127.0.0.1:6443 --port 8080:[email protected]:0 --port 8090:[email protected]:0 --port 1880:[email protected]:0 --port 5432:[email protected]:0 --port 1883:[email protected]:0 --port 8883:[email protected]:0 --port 9092:[email protected]:0 --port 46010:[email protected]:0
```
The --api-port flag is used to expose the Kubernetes API server on the host machine. If the 6443 port is already in use, you can use any other port. The --port flag is used to expose the ports of the services running in the cluster on the host machine. If any of the ports on the left side of the : is already in use, you can use any other port.

Verify that the cluster is up and running.

kubectl get nodes

The output should look like this:

NAME                                            STATUS   ROLES                  AGE   VERSION
k3d-united-manufacturing-hub-server-0           Ready    control-plane,master   10s   v1.24.4+k3s1

Install the UMH stack

Add the UMH Helm repository.

helm repo add united-manufacturing-hub https://repo.umh.app/

Update the Helm repository.
```
helm repo update
```

Create the namespace.

kubectl create namespace united-manufacturing-hub

Install the UMH stack.

helm install united-manufacturing-hub united-manufacturing-hub/united-manufacturing-hub -n united-manufacturing-hub

Open UMHLens / OpenLens on your device.
From the homepage, click on Browse Clusters in Catalog. You should see all your clusters.
Click on a cluster to connect to it.
Navigate to Helm > Releases and change the namespace from default to united-manufacturing-hub in the upper right corner.
Select the united-manufacturing-hub Release to inspect the release details, the installed resources, and the Helm values.

Troubleshooting

I don’t see the cluster in UMHLens / OpenLens

If you don’t see the cluster in UMHLens / OpenLens, you might have to add the cluster manually. To do so, follow these steps:

Open a terminal and run the following command to get the kubeconfig file:
```
k3d kubeconfig get united-manufacturing-hub
```
Copy the output of the command.
Open UMHLens / OpenLens, click on the three horizontal lines in the upper left corner and choose Files > Add Cluster.
Paste the kubeconfig and click Add clusters.

What’s next

You can follow the Getting Started guide to get familiar with the UMH stack.
If you already know your way around the United Manufacturing Hub, you can follow the Administration guides to configure the stack for production.

2 - Upgrading

This section contains the upgrading guides for the different versions the United Manufacturing Hub.

The United Manufacturing Hub is a continuously evolving product. This means that new features and bug fixes are added to the product on a regular basis. This section contains the upgrading guides for the different versions the United Manufacturing Hub.

The upgrading process is done by upgrading the Helm chart.

2.1 - Upgrade to v0.9.14

This page describes how to upgrade the United Manufacturing Hub to version 0.9.14

This page describes how to upgrade the United Manufacturing Hub to version 0.9.14. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-opcuasimulator-deployment
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-grafanaproxy
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-kafka
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect
- united-manufacturing-hub-mqttbridge
Open the Network tab.
From the Services section, delete the following services:
- united-manufacturing-hub-kafka

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

You can also change the values of the Helm chart, if needed. For example, if you want to apply the new tweaks to the resources in order to avoid the Out Of Memory crash of the MQTT Broker, you can change the following values:

iotsensorsmqtt:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
grafanaproxy:
 resources:
   requests:
     cpu: 100m
   limits:
     cpu: 300m
kafkatopostgresql:
  resources:
    requests:
      memory: 150Mi
    limits:
      memory: 300Mi
opcuasimulator:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
packmlmqttsimulator:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
tulipconnector:
  resources:
    limits:
      cpu: 30m
      memory: 50Mi
    requests:
      cpu: 10m
      memory: 20Mi
redis:
  master:
    resources:
      limits:
        cpu: 100m
        memory: 100Mi
      requests:
        cpu: 50m
        memory: 50Mi
mqtt_broker:
  resources:
    limits:
      cpu: 700m
      memory: 1700Mi
    requests:
      cpu: 300m
      memory: 1000Mi

You can also enable the new container registry by changing the values in the image or image.repository fields from unitedmanufacturinghub/<image-name> to ghcr.io/united-manufacturing-hub/<image-name>.

Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.2 - Upgrade to v0.9.13

This page describes how to upgrade the United Manufacturing Hub to version 0.9.13

This page describes how to upgrade the United Manufacturing Hub to version 0.9.13. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed.
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.3 - Upgrade to v0.9.12

This page describes how to upgrade the United Manufacturing Hub to version 0.9.12

This page describes how to upgrade the United Manufacturing Hub to version 0.9.12. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Backup RBAC configuration for MQTT Broker

This step is only needed if you enabled RBAC for the MQTT Broker and changed the default password. If you did not change the default password, you can skip this step.

Navigate to Config > ConfigMaps.
Select the united-manufacturing-hub-hivemqce-extension ConfigMap.
Copy the content of credentials.xml and save it in a safe place.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Remove MQTT Broker extension PVC

In this version we reduced the size of the MQTT Broker extension PVC. To do so, we need to delete the old PVC and create a new one. This process will set the credentials of the MQTT Broker to the default ones. If you changed the default password, you can restore them after the upgrade.

Navigate to Storage > Persistent Volume Claims.
Select the united-manufacturing-hub-hivemqce-claim-extensions PVC and click Delete.

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

There are some incompatible changes in this version. To avoid errors, you need to change the following values:

Remove property console.console.config.kafka.tls.passphrase:

console:
  console:
    config:
      kafka:
        tls:
          passphrase: "" # <- remove this line

console.extraContainers: remove the property and its content.
```
console:
  extraContainers: {} # <- remove this line
```
console.extraEnv: remove the property and its content.
```
console:
  extraEnv: "" # <- remove this line
```

console.extraEnvFrom: remove the property and its content.

console:
  extraEnvFrom: ""  # <- remove this line

console.extraVolumeMounts: remove the |- characters right after the property name. It should look like this:

console:
  extraVolumeMounts: # <- remove the `|-` characters in this line
    - name: united-manufacturing-hub-kowl-certificates
      mountPath: /SSL_certs/kafka
      readOnly: true

console.extraVolumes: remove the |- characters right after the property name. It should look like this:

console:
  extraVolumes: # <- remove the `|-` characters in this line
    - name: united-manufacturing-hub-kowl-certificates
      secret:
        secretName: united-manufacturing-hub-kowl-secrets

Change the console.service property to the following:

console:
  service:
    type: LoadBalancer
    port: 8090
    targetPort: 8080

Change the Redis URI in factoryinsight.redis:

factoryinsight:
  redis:
    URI: united-manufacturing-hub-redis-headless:6379

Set the following values in the kafka section to true, or add them if they are missing:

kafka:
  externalAccess:
    autoDiscovery:
      enabled: true
    enabled: true
  rbac:
    create: true

Change redis.architecture to standalone:
```
redis:
  architecture: standalone
```

redis.sentinel: remove the property and its content.

redis:
  sentinel: {} # <- remove all the content of this section

Remove the property redis.master.command:

redis:
  master:
  command: /run.sh # <- remove this line

timescaledb-single.fullWalPrevention: remove the property and its content.

timescaledb-single:
  fullWalPrevention:              # <- remove this line
    checkFrequency: 30            # <- remove this line
    enabled: false                # <- remove this line
    thresholds:                   # <- remove this line
      readOnlyFreeMB: 64          # <- remove this line
      readOnlyFreePercent: 5      # <- remove this line
      readWriteFreeMB: 128        # <- remove this line
      readWriteFreePercent: 8     # <- remove this line

timescaledb-single.loadBalancer: remove the property and its content.

timescaledb-single:
  loadBalancer:          # <- remove this line
    annotations:         # <- remove this line
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "4000" # <- remove this line
    enabled: true        # <- remove this line
    port: 5432           # <- remove this line

timescaledb-single.replicaLoadBalancer: remove the property and its content.

timescaledb-single:
  replicaLoadBalancer:
    annotations:         # <- remove this line
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "4000" # <- remove this line
    enabled: false       # <- remove this line
    port: 5432           # <- remove this line

timescaledb-single.secretNames: remove the property and its content.

timescaledb-single:
  secretNames: {} # <- remove this line

timescaledb-single.unsafe: remove the property and its content.

timescaledb-single:
  unsafe: false # <- remove this line

Change the value of the timescaledb-single.service.primary.type property to LoadBalancer:
```
timescaledb-single:
  service:
    primary:
      type: LoadBalancer
```

Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.4 - Upgrade to v0.9.11

This page describes how to upgrade the United Manufacturing Hub to version 0.9.11

This page describes how to upgrade the United Manufacturing Hub to version 0.9.11. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed.
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.5 - Upgrade to v0.9.10

This page describes how to upgrade the United Manufacturing Hub to version 0.9.10

This page describes how to upgrade the United Manufacturing Hub to version 0.9.10. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Grafana plugins

In this release, the Grafana version has been updated from 8.5.9 to 9.3.1. Check the release notes for further information about the changes.

Additionally, the way default plugins are installed has changed. Unfortunatly, it is necesary to manually install all the plugins that were previously installed.

If you didn’t install any plugin other than the default ones, you can skip this section.

Follow these steps to see the list of plugins installed in your cluster:

Open the browser and go to the Grafana dashboard.
Navigate to the Configuration > Plugins tab.
Select the Installed filter.
Write down all the plugins that you manually installed. You can recognize them by not having the Core tag.

The following ones are installed by default, therefore you can skip them:
- ACE.SVG by Andrew Rodgers
- Button Panel by UMH Systems Gmbh
- Button Panel by CloudSpout LLC
- Discrete by Natel Energy
- Dynamic Text by Marcus Olsson
- FlowCharting by agent
- Pareto Chart by isaozler
- Pie Chart (old) by Grafana Labs
- Timepicker Buttons Panel by williamvenner
- UMH Datasource by UMH Systems Gmbh
- Untimely by factry
- Worldmap Panel by Grafana Labs

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-grafana
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

You can also change the values of the Helm chart, if needed.

In the grafana section, find the extraInitContainers field and change its value to the following:

    - image: unitedmanufacturinghub/grafana-umh:1.1.2
      name: init-plugins
      imagePullPolicy: IfNotPresent
      command: ['sh', '-c', 'cp -r /plugins /var/lib/grafana/']
      volumeMounts:
        - name: storage
          mountPath: /var/lib/grafana

Make these changes in the kafka section:
- Set the value of the heapOpts field to -Xmx2048m -Xms2048m.
- Replace the content of the resources section with the following:
```
    limits:
      cpu: 1000m
      memory: 4Gi
    requests:
      cpu: 100m
      memory: 2560Mi
```

Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Afterwards, you can reinstall the additional Grafana plugins.

Replace VerneMQ with HiveMQ

In this upgrade we switched from using VerneMQ to HiveMQ as our MQTT Broker (you can read the blog article about it).

While this process is fully backwards compatible, we suggest to update NodeRed flows and any other additional service that uses MQTT, to use the new service broker called united-manufacturing-hub-mqtt. The old united-manufacturing-hub-vernemq is still functional and, despite the name, also points to HiveMQ, but in future upgrades will be removed.

Additionally, for production environments, we recommend to enable RBAC for the MQTT Broker.

Please double-check if all of your services can connect to the new MQTT broker. It might be needed for them to be restarted, so that they can resolve the DNS name and get the new IP. Also, it can happen with tools like chirpstack, that you need to specify the client-id as the automatically generated ID worked with VerneMQ, but is now declined by HiveMQ.

Troubleshooting

Some microservices can’t connect to the new MQTT broker

If you are using the united-manufacturing-hub-mqtt service, but some microservice can’t connect to it, restarting the microservice might solve the issue. To do so, you can delete the Pod of the microservice and let Kubernetes recreate it.

ChirpStack can’t connect to the new MQTT broker

ChirpStack uses a generated client-id to connect to the MQTT broker. This client-id is not accepted by HiveMQ. To solve this issue, you can set the client_id field in the integration.mqtt section of the chirpstack configuration file to a fixed value:

[integration]
...
  [integration.mqtt]
  client_id="chirpstack"

2.6 - Upgrade to v0.9.9

This page describes how to upgrade the United Manufacturing Hub to version 0.9.9

This page describes how to upgrade the United Manufacturing Hub to version 0.9.9. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed. In the grafana section, find the extraInitContainers field and change the value of the image field to unitedmanufacturinghub/grafana-plugin-extractor:0.1.4.
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.7 - Upgrade to v0.9.8

This page describes how to upgrade the United Manufacturing Hub to version 0.9.8

This page describes how to upgrade the United Manufacturing Hub to version 0.9.8. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed.
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.8 - Upgrade to v0.9.7

This page describes how to upgrade the United Manufacturing Hub to version 0.9.7

This page describes how to upgrade the United Manufacturing Hub to version 0.9.7. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

You can also change the values of the Helm chart, if needed.

Make these changes in the grafana section:

Replace the content of datasources with the following:

    datasources.yaml:
      apiVersion: 1
      datasources:
      - access: proxy
        editable: false
        isDefault: true
        jsonData:
          apiKey: $FACTORYINSIGHT_PASSWORD
          apiKeyConfigured: true
          customerId: $FACTORYINSIGHT_CUSTOMERID
          serverURL: http://united-manufacturing-hub-factoryinsight-service/
        name: umh-datasource
        orgId: 1
        type: umh-datasource
        url: http://united-manufacturing-hub-factoryinsight-service/
        version: 1
      - access: proxy
        editable: false
        isDefault: false
        jsonData:
          apiKey: $FACTORYINSIGHT_PASSWORD
          apiKeyConfigured: true
          baseURL: http://united-manufacturing-hub-factoryinsight-service/
          customerID: $FACTORYINSIGHT_CUSTOMERID
        name: umh-v2-datasource
        orgId: 1
        type: umh-v2-datasource
        url: http://united-manufacturing-hub-factoryinsight-service/
        version: 1

Replace the content of env with the following:

    GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: umh-datasource,umh-factoryinput-panel,umh-v2-datasource

Replace the content of extraInitContainers with the following:

  - name: init-umh-datasource
    image: unitedmanufacturinghub/grafana-plugin-extractor:0.1.3
    volumeMounts:
    - name: storage
      mountPath: /var/lib/grafana
    imagePullPolicy: IfNotPresent

In the timescaledb-single section, make sure that the image.tag field is set to pg13.8-ts2.8.0-p1.

Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Change Factoryinsight API version

The Factoryinsight API version has changed from v1 to v2. To make sure that you are using the new version, click on any Factoryinsight Pod and check that the VERSION environment variable is set to 2.

If it’s not, follow these steps:

Navigate to the Workloads > Deployments tab.
Select the united-manufacturing-hub-factoryinsight-deployment deployment.
Click the Edit button to open the deployment’s configuration.
Find the spec.template.spec.containers[0].env field.
Set the value field of the VERSION variable to 2.

2.9 - Upgrade to v0.9.6

This page describes how to upgrade the United Manufacturing Hub to version 0.9.6

This page describes how to upgrade the United Manufacturing Hub to version 0.9.6. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Add new index to the database

In this version, a new index has been added to the processValueTabe table, allowing to speed up the queries.

Open a shell in the database

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Create the index

Execute the following query:

CREATE INDEX ON processvaluetable(valuename, asset_id) WITH (timescaledb.transaction_per_chunk);
REINDEX TABLE processvaluetable;

This command could take a while to complete, especially on larger tables.

Type exit to close the shell.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed.
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

2.10 - Upgrade to v0.9.5

This page describes how to upgrade the United Manufacturing Hub to version 0.9.5

This page describes how to upgrade the United Manufacturing Hub to version 0.9.5. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Alter ordertable constraint

In this version, one of the constraints of the ordertable table has been modified.

Make sure to backup the database before exectuing the following steps.

Open a shell in the database

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Alter the table

Check for possible conflicts in the ordertable table:

SELECT order_name, asset_id, count(*) FROM ordertable GROUP BY order_name, asset_id HAVING count(*) > 1;

If the result is empty, you can skip the next step.

Delete the duplicates:

DELETE FROM ordertable ox USING (
     SELECT MIN(CTID) as ctid, order_name, asset_id
     FROM ordertable
     GROUP BY order_name, asset_id HAVING count(*) > 1
     ) b
WHERE ox.order_name = b.order_name AND ox.asset_id = b.asset_id
AND ox.CTID <> b.ctid;

If the data cannot be deleted, you have to manually update each duplicate order_names to a unique value.

Get the name of the constraint:

SELECT conname FROM pg_constraint WHERE conrelid = 'ordertable'::regclass AND contype = 'u';

Drop the constraint:

ALTER TABLE ordertable DROP CONSTRAINT ordertable_asset_id_order_id_key;

Add the new constraint:

ALTER TABLE ordertable ADD CONSTRAINT ordertable_asset_id_order_name_key UNIQUE (asset_id, order_name);

Now you can close the shell by typing exit and continue with the upgrade process.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
You can also change the values of the Helm chart, if needed.
- Enable the startup probe for the Kafka Broker by adding the following into the kafka section:
```
startupProbe:
  enabled: true
  failureThreshold: 600
  periodSeconds: 10
  timeoutSeconds: 10
```
Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Changes to the messages

Some messages have been modified in this version. You need to update some payolads in your Node-RED flows.

modifyState:
- start_time_stamp has been renamed to timestamp_ms
- end_time_stamp has been renamed to timestamp_ms_end
modifyProducedPieces:
- start_time_stamp has been renamed to timestamp_ms
- end_time_stamp has been renamed to timestamp_ms_end
deleteShiftByAssetIdAndBeginTimestamp and deleteShiftById have been removed. Use the deleteShift message instead.

2.11 - Upgrade to v0.9.4

This page describes how to upgrade the United Manufacturing Hub to version 0.9.4

This page describes how to upgrade the United Manufacturing Hub to version 0.9.4. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Helm repo name: united-manufacturing-hub
URL: https://repo.umh.app

Then click Add.

Clear Workloads

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

Open the Workloads tab.
From the Deployment section, delete the following deployments:
- united-manufacturing-hub-barcodereader
- united-manufacturing-hub-factoryinsight-deployment
- united-manufacturing-hub-kafkatopostgresql
- united-manufacturing-hub-mqttkafkabridge
- united-manufacturing-hub-iotsensorsmqtt
- united-manufacturing-hub-opcuasimulator-deployment
From the StatefulSet section, delete the following statefulsets:
- united-manufacturing-hub-mqttbridge
- united-manufacturing-hub-hivemqce
- united-manufacturing-hub-nodered
- united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

Navigate to the Helm > Releases tab.
Select the united-manufacturing-hub release and click Upgrade.
In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

You can also change the values of the Helm chart, if needed.

If you have enabled the Kafka Bridge, find the section _000_commonConfig.kafkaBridge.topicmap and set the value to the following:

- bidirectional: false
  name: HighIntegrity
  send_direction: to_remote
  topic: ^ia\.(([^r.](\d|-|\w)*)|(r[b-z](\d|-|\w)*)|(ra[^w]))\.(\d|-|\w|_)+\.(\d|-|\w|_)+\.((addMaintenanceActivity)|(addOrder)|(addParentToChild)|(addProduct)|(addShift)|(count)|(deleteShiftByAssetIdAndBeginTimestamp)|(deleteShiftById)|(endOrder)|(modifyProducedPieces)|(modifyState)|(productTag)|(productTagString)|(recommendation)|(scrapCount)|(startOrder)|(state)|(uniqueProduct)|(scrapUniqueProduct))$
- bidirectional: false
  name: HighThroughput
  send_direction: to_remote
  topic: ^ia\.(([^r.](\d|-|\w)*)|(r[b-z](\d|-|\w)*)|(ra[^w]))\.(\d|-|\w|_)+\.(\d|-|\w|_)+\.(process[V|v]alue).*$

For more information, see the Kafka Bridge configuration

If you have enabled Barcodereader, find the barcodereader section and set the following values, adding the missing ones and updating the already existing ones:

enabled: false
image:
  pullPolicy: IfNotPresent
resources:
  requests:
    cpu: "2m"
    memory: "30Mi"
  limits:
    cpu: "10m"
    memory: "60Mi"
scanOnly: false # Debug mode, will not send data to kafka

Click Upgrade.

The upgrade process can take a few minutes. The process is complete when the Status field of the release is Deployed.

3 - Administration

This section describes how to manage and configure the United Manufacturing Hub cluster.

In this section, you will find information about how to manage and configure the United Manufacturing Hub cluster, from customizing the cluster to access the different services.

3.1 - Access the Database

This page describes how to access the United Manufacturing Hub database to perform SQL operations using a database client, the CLI or Grafana.

There are multiple ways to access the database. If you want to just visualize data, then using Grafana or a database client is the easiest way. If you need to also perform SQL commands, then using a database client or the CLI are the best options.

Generally, using a database client gives you the most flexibility, since you can both visualize the data and manipulate the database. However, it requires you to install a database client on your machine.

Using the CLI gives you more control over the database, but it requires you to have a good understanding of SQL.

Grafana, on the other hand, is for visualizing data. It is a good option if you just want to see the data in a dashboard and don’t need to manupulate it.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Get the database credentials

If you are not using the CLI, you need to know the database credentials. You can find them in the timescale-post-init-pw Secret. By default, the username is factoryinsight and the password is changeme.

...
ALTER USER factoryinsight WITH PASSWORD 'changeme';
...

Access the database using a database client

There are many database clients that you can use to access the database. Here’s a list of some of the most popular database clients:

Database clients
Name	Free or Paid	Platforms
pgAdmin	Free	Windows, macOS, Linux
DataGrip	Paid	Windows, macOS, Linux
DBeaver	Both	Windows, macOS, Linux

For the sake of this tutorial, pgAdmin will be used as an example, but other clients have similar functionality. Refer to the specific client documentation for more information.

Forward the database port to your local machine

From the Pods section in UMHLens / OpenLens, find the united-manufacturing-hub-timescaledb-0 Pod.
In the Pod Details window, click the Forward button next to the postgresql:5432/TCP port.
Enter a port number, such as 5432, and click Start. You can disable the Open in browser option if you don’t want to open the port in your browser.

Using pgAdmin

You can use pgAdmin to access the database. To do so, you need to install the pgAdmin client on your machine. For more information, see the pgAdmin documentation.

Once you have installed the client, you can add a new server from the main window.
In the General tab, give the server a meaningful name. In the Connection tab, enter the database credentials:
- The Host name/address is localhost.
- The Port is the port you forwarded.
- The Maintenance database is postgres.
- The Username and Password are the ones you found in the Secret.
Click Save to save the server.

You can now connect to the database by double-clicking the server.

Use the side menu to navigate through the server. The tables are listed under the Schemas > public > Tables section of the factoryinsight database.

Refer to the pgAdmin documentation for more information on how to use the client to perform database operations.

Access the database using the command line interface

You can access the database from the command line using the psql command directly from the united-manufacturing-hub-timescaledb-0 Pod.

You will not need credentials to access the database from the Pod’s CLI.

Open a shell in the database Pod

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Perform SQL commands

Once you have a shell in the database, you can perform SQL commands.

For example, to create an index on the processValueTable:
```
CREATE INDEX ON processvaluetable (valuename);
```
When you are done, exit the postgres shell:
```
 exit
```

Access the database using Grafana

You can use Grafana to visualize data from the database.

Add PostgreSQL as a data source

Open the Grafana dashboard in your browser.
From the Configuration (gear) icon, select Data Sources.
Click Add data source and select PostgreSQL.
Configure the connection to the database:
- The Host is united-manufacturing-hub.united-manufacturing-hub.svc.cluster.local:5432.
- The Database is factoryinsight.
- The User and Password are the ones you found in the Secret.
- Set TLS/SSL Mode to require.
- Enable TimescaleDB.
Everything else can be left as the default.
Click Save & Test to save the data source.
Now click on Explore to start querying the database.
You can also create dashboards using the newly created data source.

What’s next

See a list of SQL commands
See how to Delete Assets from the Database
See how to Reduce the Database Size
See how to Backup and Restore the Database
See how to Expose Grafana to the Internet

3.2 - Access Services From Within the Cluster

This page describes how to access services from within the cluster.

All the services deployed in the cluster are visible to each other. That makes it easy to connect them together.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Connect to a service from another service

To connect to a service from another service, you can use the service name as the host name.

To get a list of available services and related ports you can open UMHLens / OpenLens and go to Network > Services.

All of them are available from within the cluster. The ones of type LoadBalancer are also available from outside the cluster using the node IP.

Example

The most common use case is to connect to the MQTT Broker from Node-RED.

To do that, when you create the MQTT node, you can use the service name united-manufacturing-hub-mqtt as the host name and one the ports listed in the Ports column.

The MQTT service name has changed since version 0.9.10. If you are using an older version, use united-manufacturing-hub-vernemq instead of united-manufacturing-hub-mqtt.

What’s next

3.3 - Access Services Outside the Cluster

This page describe how to access services from outside the cluster.

Some of the microservices in the United Manufacturing Hub are exposed outside the cluster with a LoadBalancer service. A LoadBalancer is a service that exposes a set of Pods on the same network as the cluster, but not necessarily to the entire internet. The LoadBalancer service provides a single IP address that can be used to access the Pods.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Accessing the services

The LoadBalancer service provides a single IP address that can be used to access the Pods. To find the IP address, open UMHLens / OpenLens and navigate to Network > Services. The IP address is listed in the External IP column.

To access the services, use the IP address and the port number of the service, e.g. http://192.168.1.100:8080.

If you installed the United Manufacturing Hub on your local machine, either using the Management Console or the command line, the services are accessible at localhost:<port-number>.

Services with LoadBalancer by default

The following services are exposed outside the cluster with a LoadBalancer service by default:

Database at port 5432
Kafka Console at port 8090
Kafka Broker at port 9094
Grafana at port 8080
MQTT Broker at port 1883
Node-RED at port 1880
OPCUA Simulator at port 46010

To access Node-RED, you need to use the /node-red path, e.g. http://192.168.1.100:1880/node-red.

Services without a LoadBalancer

Some of the microservices in the United Manufacturing Hub are exposed via a ClusterIP service. That means that they are only accessible from within the cluster itself. To access them from outside the cluster, you need to create a LoadBalancer service.

Create a LoadBalancer service

If you are looking to expose th Kafka broker, follow the instructions in the Access Kafka outside the cluster page.

For any other microservice, follow these steps to enable the LoadBalancer service:

Open UMHLens / OpenLens and navigate to Network > Services.
Select the service and click the Edit button.
Scroll down to the status.loadBalancer section and change it to the following:
```
status:
  loadBalancer:
    ingress:
    - ip: <external-ip>
```
Replace <external-ip> with the external IP address of the node.
Scroll to the spec.type section and change the value from ClusterIP to LoadBalancer.
Click Save to apply the changes.

If you installed the United Manufacturing Hub on your local machine, either using the Management Console or the command line, you also need to map the port exposed by the k3d cluster to a port on your local machine. To do that, run the following command:

k3d cluster edit united-manufacturing-hub --port-add "<local-port>:<cluster-port>@server:0"

Replace <local-port> with a free port number on your local machine, and <cluster-port> with the port number of the service.

Port forwarding in UMHLens / OpenLens

If you don’t want to create a LoadBalancer service, effectively exposing the microservice to anyone that has access to the host IP address, you can use UMHLens / OpenLens to forward the port to your local machine.

Open UMHLens / OpenLens and navigate to Network > Services.
Select the service that you want to access.
Scroll down to the Connection section and click the Forward… button.
From the dialog, you can choose a port on your local machine to forward the cluster port from, or you can leave it empty to use a random port.
Click Forward to apply the changes.
If you left the checkbox Open in browser checked, then the service will open in your default browser.

You can see and manage the forwarded ports of your cluster in the Network > Port Forwarding section.

Port forwarding can be unstable, especially if the connection to the cluster is slow. If you are experiencing issues, try to create a LoadBalancer service instead.

Security considerations

MQTT broker

There are some security considerations to keep in mind when exposing the MQTT broker.

By default, the MQTT broker is configured to allow anonymous connections. This means that anyone can connect to the broker without providing any credentials. This is not recommended for production environments.

To secure the MQTT broker, you can configure it to require authentication. For that, you can either enable RBAC or set up HiveMQ PKI (recommended for production environments).

If you are using a version of the United Manufacturing Hub older than 0.9.10, then you need to change the ACL configuration to allow your MQTT client to connect to the broker.

Troubleshooting

LoadBalancer service stuck in Pending state

If the LoadBalancer service is stuck in the Pending state, it probably means that the host port is already in use. To fix this, edit the service and change the section spec.ports.port to a different port number.

What’s next

3.4 - Access Kafka Outside the Cluster

This page describes how to access Kafka from outside the cluster.

By default the Kafka broker is only available from within the cluster, therefore you cannot access it from external applications.

You can enable external access from the Kafka configuration.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Enable external access from Kafka configuration

From UMHLens / OpenLens, go to Helm > Releases.
Click on the Upgrade button.

Search for the kafka section and edit the following values:

...
kafka:
...
  externalAccess:
    autoDiscovery:
      enabled: true
    ...
    enabled: true
  ...
  rbac:
    create: true
...

Click Upgrade.

To verify that the LoadBalancer service is created, go to Network > Services and search for united-manufacturing-hub-kafka-external.

Now you can connect to Kafka from external applications using the node IP and the port 9094.

What’s next

3.5 - Expose Grafana to the Internet

This page describes how to expose Grafana to the Internet.

This page describes how to expose Grafana to the Internet so that you can access it from outside the Kubernetes cluster.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Enable the ingress

To expose Grafana to the Internet, you need to enable the ingress.

Open UMHLens / OpenLens and go to the Helm > Releases page.
Click the Upgrade button and search for Grafana.
Scroll down to the ingress section
Set the enabled field to true.
Add you domain name to the hosts field.
Click Upgrade to apply the changes.

Remember to add a DNS record for your domain name that points to the external IP address of the Kubernetes host. You can find the external IP address of the Kubernetes host on the Nodes page in UMHLens / OpenLens.

What’s next

See how to Access Factoryinsight Outside the Cluster

3.6 - Install Custom Drivers in NodeRed

This page describes how to install custom drivers in NodeRed.

NodeRed is running on Alpine Linux as non-root user. This means that you can’t install packages with apk. This tutorial shows you how to install packages with proper security measures.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Change the security context

From the StatefulSets section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered to open the details page.
Click the Edit button to open the StatefulSet’s configuration.
Press Ctrl+F and search for securityContext.

Set the values of the runAsUser field to 0, of fsGroup to 0, and of runAsNonRoot to false.

...
       securityContext:
         runAsUser: 0
   restartPolicy: Always
   terminationGracePeriodSeconds: 30
   dnsPolicy: ClusterFirst
   securityContext:
     runAsUser: 0
     runAsNonRoot: false
     fsGroup: 0
...

Click Save.

Install the packages

From the Pods section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Install the packages with apk:
```
apk add <package>
```
For example, to install unixodbc:
```
apk add unixodbc
```
You can find the list of available packages here.
Exit the shell.

Revert the security context

For security reasons, you should revert the security context after you install the packages.

From the StatefulSets section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered to open the details page.
Click the Edit button to open the StatefulSet’s configuration.

Set the values of the runAsUser field to 1000, of fsGroup to 1000, and of runAsNonRoot to true.

...
       securityContext:
         runAsUser: 1000
   restartPolicy: Always
   terminationGracePeriodSeconds: 30
   dnsPolicy: ClusterFirst
   securityContext:
     runAsUser: 1000
     runAsNonRoot: true
     fsGroup: 1000
...

Click Save.

What’s next

3.7 - Execute Kafka Shell Scripts

This page describes how to execute Kafka shell scripts.

When working with Kafka, you may need to execute shell scripts to perform administrative tasks. This page describes how to execute Kafka shell scripts.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open a shell in the Kafka container

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-kafka-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Navigate to the Kafka bin directory:
```
cd /opt/bitnami/kafka/bin
```
Execute any Kafka shell scripts. For example, to list all topics:
```
./kafka-topics.sh --list --zookeeper zookeeper:2181
```
To exit the shell:
```
exit
```

What’s next

See Access Kafka outside the cluster.

3.8 - Reduce database size

This page describes how to reduce the size of the United Manufacturing Hub database.

Over time, time-series data can consume a large amount of disk space. To reduce the amount of disk space used by time-series data, there are three options:

Enable data compression. This reduces the required disk space by applying mathematical compression to the data. This compression is lossless, so the data is not changed in any way. However, it will take more time to compress and decompress the data. For more information, see how TimescaleDB compression works.
Enable data retention. This deletes old data that is no longer needed, by setting policies that automatically delete data older than a specified time. This can be beneficial for managing the size of the database, as well as adhering to data retention regulations. However, by definition, data loss will occur. For more information, see how TimescaleDB data retention works.
Downsampling. This is a method of reducing the amount of data stored by aggregating data points over a period of time. For example, you can aggregate data points over a 30-minute period, instead of storing each data point. If exact data is not required, downsampling can be useful to reduce database size. However, data may be less accurate.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open the database shell

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Enable data compression

To enable data compression, you need to execute the following SQL command from the database shell:

SELECT add_retention_policy('processvaluetable', INTERVAL '7 days');

This command will set a retention policy on the processvaluetable table, which will delete data older than 7 days.

Enable data retention

To enable data retention, you need to execute the following SQL command from the database shell:

SELECT add_compression_policy('processvaluetable', INTERVAL '7 days');

This command will set a compression policy on the processvaluetable table, which will compress data older than 7 days.

What’s next

3.9 - Delete Assets from the Database

This task shows you how to delete assets from the database.

This is useful if you have created assets by mistake, or to delete the ones that are no longer needed.

This task deletes data from the database. Make sure you have a backup of the database before you proceed.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Also, make sure to backup the database before you proceed. For more information, see Backing Up and Restoring the Database.

Open the database shell

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Choose the assets to delete

You have multiple choices to delete assets, like deleting a single asset, or deleting all assets in a location, or deleting all assets with a specific name.

To do so, you can customize the SQL command using different filters. Specifically, a combination of the following filters:

assetname
location
customer

To filter an SQL command, you can use the WHERE clause. For example, using all of the filters:

WHERE assetname = <my-asset> AND location = <my-location> AND customer = <my-customer>;

You can use any combination of the filters, even just one of them.

Here are some examples:

Delete all assets with the same name from any location and any customer:
```
WHERE assetname = '<asset-name>'
```
Delete all assets in a specific location:
```
 WHERE location = '<location-name>'
```

Delete all assets with the same name in a specific location:

WHERE assetname = '<asset-name>' AND location = '<location-name>'

Delete all assets with the same name in a specific location for a single customer:

WHERE assetname = 'my-asset' AND location = 'my-location' AND customer = 'customer'

Delete the assets

Once you know the filters you want to use, you can use the following SQL commands to delete assets:

BEGIN;

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM shifttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM counttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM ordertable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM processvaluestringtable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM processvaluetable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM producttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM shifttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM statetable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM assettable WHERE id IN (SELECT id FROM assets_to_be_deleted);

COMMIT;

Optionally, you can add the following code before the last WITH statement if you used the track&trace feature:

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM producttagtable WHERE product_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM producttagstringtable WHERE product_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM productinheritancetable WHERE parent_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted) OR child_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

What’s next

See Backing Up and Restoring the Database

3.10 - Change the Language in Factoryinsight

This page describes how to change the language in Factoryinsight, in order to display the returned text in a different language.

You can change the language in Factoryinsight if you want to localize the returned text, like stop codes, to a different language.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Access the database shell

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Change the language

Execute the following command to change the language:

INSERT INTO configurationtable (customer, languagecode) VALUES ('factoryinsight', <code>) ON CONFLICT(customer) DO UPDATE SET languagecode=<code>;

where <code> is the language code. For example, to change the language to German, use 0.

Supported languages

Factoryinsight supports the following languages:

Supported languages
Language	Code
German	0
English	1
Turkish	2

What’s next

See Access the Database

3.11 - Explore Cached Data

This page shows how to explore cached data in the United Manufacturing Hub.

When working with the United Manufacturing Hub, you might want to visualize information about the cached data. This page shows how you can access the cache and explore the data.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open a shell in the cache Pod

Open UMHLens / OpenLens and navigate to the Config > Secrets page.
Get the cache password from the Secret redis-secret.
From the Pods section click on united-manufacturing-hub-redis-master-0 to open the details page.
If you have multiple cache Pods, you can select any of them.
Click the Pod Shell button to open a shell in the container.
Enter the shell:
```
redis-cli -a <cache-password>
```
Now you can execute any command. For example, to get the number of keys in the cache, run:
```
KEYS *
```
Or, to get the cache size, run:
```
DBSIZE
```

For more information about Redis commands, see the Redis documentation.

What’s next

Access the database

3.12 - Optimize Time Consuming Queries

This page shows how to optimize the database in order to reduce the time needed to execute queries.

When you have a large database, it is possible that some queries take a long time to execute. This especially shows when you are using Grafana and the dropdown menu in the datasource takes a long time to load or does not load at all.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Your United Manufacturing Hub must be at or later than version 0.9.4. To check the United Manufacturing Hub version, open UMHLens / OpenLens and go to Helm > Releases. The version is listed in the Version column.

Open a shell in the database container

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Create an index

Indexes are used to speed up queries. Run this query to create an index on the processvaluetable table:

CREATE INDEX ON processvaluetable(valuename, asset_id) WITH (timescaledb.transaction_per_chunk);

Rollback factoryinsight

If you have already created an index, you can rollback the factoryinsight deployment to version 0.9.4. This way it will use a less optimized but faster query, significantly reducing the execution time.

From the Deployments section in UMHLens / OpenLens, click on united-manufacturing-hub-factoryinsight-deployment to open the details page.
Click the Edit button to open the deployment’s configuration.
Scroll down to the spec.containers section and change the image value to unitedmanufacturinghub/factoryinsight:0.9.4.
Click Save.

What’s next

See Access the Database
See Reduce Database Size

3.13 - Optimize Database Datatypes

This page describes how to change the datatype of some columns in the database in order to optimize the performance.

In version 0.9.5 and prior, some tables in the database were created with the varchar data type. This data type is not optimal for storing large amounts of data. In version 0.9.6, the data type of some columns was changed from varchar to text. This migration optimizes the database, by changing the data type of some columns from varchar to text.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Alter the tables

Execute the following SQL statements:

ALTER TABLE assettable ALTER COLUMN assetid TYPE text;
ALTER TABLE assettable ALTER COLUMN location TYPE text;
ALTER TABLE assettable ALTER COLUMN customer TYPE text;
ALTER TABLE producttable ALTER COLUMN product_name TYPE text;
ALTER TABLE ordertable ALTER COLUMN order_name TYPE text;
ALTER TABLE configurationtable ALTER COLUMN customer TYPE text;
ALTER TABLE componenttable ALTER COLUMN componentname TYPE text;

Then confirm the changes by using the following SQL statements:

SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'assettable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'producttable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'ordertable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'configurationtable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'componenttable';

4 - Backup & Recovery

This section contains information about how to backup and recover various components of the United Manufacturing Hub.

4.1 - Backup and Restore the United Manufacturing Hub

This page describes how to backup and restore the entire United Manufacturing Hub.

This page describes how to back up the following:

All Node-RED flows
All Grafana dashboards
The Helm values used for installing the united-manufacturing-hub release
All the contents of the United Manufacturing Hub database

It does not back up:

Additional databases other than the United Manufacturing Hub default database
TimescaleDB continuous aggregates: Follow the official documentation to learn how.
TimescaleDB policies: Follow the official documentation to learn how.
Everything else not included in the previous list

This procedure only works on Windows.

Before you begin

Download the backup scripts and extract the content in a folder of your choice.

For this task, you need to have PostgreSQL installed on your machine.

You also need to have enough space on your machine to store the backup. To check the size of the database, follow the steps below:

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Run the following command to get the size of the database:

SELECT pg_size_pretty(pg_database_size('factoryinsight'));

Backup

Generate Grafana API Key

Create a Grafana API Token for an admin user by following these steps:

Open the Grafana UI in your browser and log in with an admin user.
Click on the Settings icon in the left sidebar and select API Keys.
Give the API key a name and change its role to Admin.
Optionally set an expiration date.
Click Add.
Copy the generated API key and save it for later.

Stop workloads

To prevent data inconsistencies, you need to temporarily stop the MQTT and Kafka Brokers.

In UMHLens / OpenLens go to the Workloads > StatefulSets tab.
Select the united-manufacturing-hub-kafka StatefulSet
Click the Scale button to select the number of replicas for the StatefulSet.
Set the number of replicas to 0 and click Scale.
Repeat the process for the united-manufacturing-hub-hivemqce StatefulSet.

Backup using the script

The backup script is located inside the folder you downloaded earlier.

Open a terminal and navigate inside the folder.
```
cd <FOLDER_PATH>
```
Run the script:
```
.\backup.ps1 -IP <IP_OF_THE_SERVER> -GrafanaToken <GRAFANA_API_KEY> -KubeconfigPath <PATH_TO_KUBECONFIG>
```
You can find a list of all available parameters down below.
If OutputPath is not set, the backup will be stored in the current folder.

This script might take a while to finish, depending on the size of your database and your connection speed.

If the connection is interrupted, there is currently no option to resume the process, therefore you will need to start again.

Here is a list of all available parameters:

Available parameters
Parameter	Description	Required	Default value
`GrafanaToken`	Grafana API key	Yes
`IP`	IP of the cluster to backup	Yes
`KubeconfigPath`	Path to the kubeconfig file	Yes
`DatabaseDatabase`	Name of the databse to backup	No	factoryinsight
`DatabasePassword`	Password of the database user	No	changeme
`DatabasePort`	Port of the database	No	5432
`DatabaseUser`	Database user	No	factoryinsight
`DaysPerJob`	Number of days worth of data to backup in each parallel job	No	31
`EnableGpgEncryption`	Set to `true` if you want to encrypt the backup	No	`false`
`EnableGpgSigning`	Set to `true` if you want to sign the backup	No	`false`
`GpgEncryptionKeyId`	ID of the GPG key used for encryption	No
`GpgSigningKeyId`	ID of the GPG key used for signing	No
`GrafanaPort`	External port of the Grafana service	No	8080
`OutputPath`	Path to the folder where the backup will be stored	No	Current folder
`ParallelJobs`	Number of parallel job backups to run	No	4
`SkipDiskSpaceCheck`	Skip checking available disk space	No	`false`
`SkipGpgQuestions`	Set to `true` if you want to sign or encrypt the backup	No	`false`

Restore

Each component of the United Manufacturing Hub can be restored separately, in order to allow for more flexibility and to reduce the damage in case of a failure.

Cluster configuration

To restore the Kubernetes cluster, execute the .\restore-helm.ps1 script with the following parameters:

.\restore-helm.ps1 -KubeconfigPath <PATH_TO_KUBECONFIG> -BackupPath <PATH_TO_BACKUP_FOLDER>

Verify that the cluster is up and running by opening UMHLens / OpenLens and checking if the workloads are running.

Grafana dashboards

To restore the Grafana dashboards, you first need to create a Grafana API Key for an admin user in the new cluster by following these steps:

Open the Grafana UI in your browser and log in with an admin user.
Click on the Settings icon in the left sidebar and select API Keys.
Give the API key a name and change its role to Admin.
Optionally set an expiration date.
Click Add.
Copy the generated API key and save it for later.

Then, on your local machine, execute the .\restore-grafana.ps1 script with the following parameters:

.\restore-grafana.ps1 -FullUrl http://<IP_OF_THE_SERVER>:8080 -Token <GRAFANA_API_KEY> -BackupPath <PATH_TO_BACKUP_FOLDER>

Restore Node-RED flows

To restore the Node-RED flows, execute the .\restore-nodered.ps1 script with the following parameters:

.\restore-nodered.ps1 -KubeconfigPath <PATH_TO_KUBECONFIG> -BackupPath <PATH_TO_BACKUP_FOLDER>

Restore the database

To restore the database, execute the .\restore-timescale.ps1 script with the following parameters:

.\restore-timescale.ps1 -Ip <IP_OF_THE_SERVER> -BackupPath <PATH_TO_BACKUP_FOLDER> -PatroniSuperUserPassword <DATABASE_PASSWORD>

What’s next

Take a look at the UMH-Backup repository
Learn how to manually backup and restore the database
Read how to import and export Node-RED flows via the UI

4.2 - Backup and Restore Database

This page describes how to backup and restore the database.

Before you begin

For this task, you need to have PostgreSQL installed on your machine.

You also need to have enough space on your machine to store the backup. To check the size of the database, follow the steps below:

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Run the following command to get the size of the database:

SELECT pg_size_pretty(pg_database_size('factoryinsight'));

Backing up the database

Follow these steps to create a backup of the factoryinsight database on your machine:

Open a terminal, and using the cd command, navigate to the folder where you want to store the backup. For example:
```
   cd C:\Users\user\backups
   
```
```
   cd /Users/user/backups
   
```
```
   cd /home/user/backups
   
```
If the folder does not exist, you can create it using the mkdir command or your file manager.
Run the following command:
```
pg_dump -h <REMOTE_HOST> -p 5432 -U factoryinsight -Fc -f <BACKUP_NAME>.bak factoryinsight
```
- <REMOTE_HOST> is the IP of the server where the database is running. Use localhost if you installed the United Manufacturing Hub using k3d.
- <BACKUP_NAME> is the name of the backup file.

Grafana database

If you want to backup the Grafana database, you can follow the same steps as above, but you need to replace any occurence of factoryinsight with grafana.

Additionally, you also need to write down the credentials in the grafana-secret Secret, as they will be needed to access the dashboard after restoring the database.

Restoring the database

This section is untested. Please report any issues you encounter.

For this section, we assume that you are restoring the data to a fresh United Manufacturing Hub installation with an empty database.

Copy the backup file to the database pod

Open UMHLens / OpenLens.
Launch a new terminal sesstion by clicking on the + button in the bottom-left corner of the window.
Run the following command to copy the backup file to the database pod:
```
kubectl cp /path/to/local/backup.bak united-manufacturing-hub-timescaledb-0:/tmp/backup.bak
```
Replace /path/to/local/backup.bak with the path to the backup file on your machine.

This step could take a while depending on the size of the backup file.

Temporarly disable kafkatopostrgesql

Navigate to Workloads > Deployments.
Select the united-manufacturing-hub-kafkatopostgresql Deployment.
Click the Scale button to select the number of replicas for the Deployment.
Scale the number of replicas to 0.

Open a shell in the database pod

From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.
Click the Pod Shell button to open a shell in the container.
Enter the postgres shell:
```
psql
```
Connect to the database:
```
\c factoryinsight
```

Restore the database

Drop the existing database:
```
DROP DATABASE factoryinsight;
```

Create a new database:

CREATE DATABASE factoryinsight;
\c factoryinsight
CREATE EXTENSION IF NOT EXISTS timescaledb;

Put the database in maintenance mode:
```
SELECT timescaledb_pre_restore();
```

Restore the database:

\! pg_restore -Fc -d factoryinsight /tmp/backup.bak

Take the database out of maintenance mode:
```
SELECT timescaledb_post_restore();
```

Enable kafkatopostgresql

Navigate to Workloads > Deployments.
Select the united-manufacturing-hub-kafkatopostgresql Deployment.
Click the Scale button to select the number of replicas for the Deployment.
Scale the number of replicas to the original value, usually 1.

What’s next

See the official TimescaleDB backup guide
See the official pg_dump documentation

4.3 - Import and Export Node-RED Flows

This page describes how to import and export Node-RED flows.

Export Node-RED Flows

To export Node-RED flows, please follow the steps below:

Access Node-RED by navigating to http://<CLUSTER-IP>:1880/nodered in your browser. Replace <CLUSTER-IP> with the IP address of your cluster, or localhost if you are running the cluster locally.
From the top-right menu, select Export.
From the Export dialog, select wich nodes or flows you want to export.
Click Download to download the exported flows, or Copy to clipboard to copy the exported flows to the clipboard.

The credentials of the connector nodes are not exported. You will need to re-enter them after importing the flows.

Import Node-RED Flows

To import Node-RED flows, please follow the steps below:

Access Node-RED by navigating to http://<CLUSTER-IP>:1880/nodered in your browser. Replace <CLUSTER-IP> with the IP address of your cluster, or localhost if you are running the cluster locally.
From the top-right menu, select Import.
From the Import dialog, select the file containing the exported flows, or paste the exported flows from the clipboard.
Click Import to import the flows.

5 - Security

This section contains information about how to secure the United Manufacturing Hub.

5.1 - Change VerneMQ ACL Configuration

This page describes how to change the ACL configuration to allow more users to publish to the MQTT broker

Change VerneMQ ACL configuration

Open UMHLens / OpenLens
Navigate to Helm > Releases.
Select the united-manufacturing-hub release and click Upgrade.
Find the _000_commonConfig.infrastrucutre.mqtt section.
Update the AclConfig value to allow unrestricted access, for example:
```
AclConfig: |
  pattern # allow all  
```
Click Upgrade to apply the changes.

What’s next

You can find more information about the ACL configuration in the VerneMQ documentation.

5.2 - Enable RBAC for the MQTT Broker

This page describes how to enable Role-Based Access Control (RBAC) for the MQTT broker.

Enable RBAC

Open UMHLens / OpenLens
Navigate to Helm > Releases.
Select the united-manufacturing-hub release and click Upgrade.
Find the mqtt_broker section.
Locate the rbacEnabled parameter and change its value from false to true.
Click Upgrade.

Now all MQTT connections require password authentication with the following defaults:

Username: node-red
Password: INSECURE_INSECURE_INSECURE

Change default credentials

Open UMHLens / OpenLens
Navigate to Workloads > Pods.
Select the united-manufacturing-hub-hivemqce-0 Pod.
Click the Pod Shell button to open a shell in the container.
Navigate to the installation directory of the RBAC extension.
```
cd extensions/hivemq-file-rbac-extension/
```
Generate a password hash with this command.
```
java -jar hivemq-file-rbac-extension-<version>.jar -p <password>
```
- Replace <version> with the version of the HiveMQ CE extension. If you are not sure which version is installed, you can press Tab after typing java -jar hivemq-file-rbac-extension- to autocomplete the version.
- Replace <password> with your desired password. Do not use any whitespaces.
Copy the output of the command. It should look similar to this:
```
$2a$10$Q8ZQ8ZQ8ZQ8ZQ8ZQ8ZQ8Zu
```
Navigate to Config > ConfigMaps.
Select the united-manufacturing-hub-hivemqce-extension ConfigMap.
Click the Edit button to open the ConfigMap editor.
In the data.credentials.xml section, replace the strings inbetween the <password> tags with the password hash generated in step 7.
You can use a different password for each different microservice. Just remember that you will need to update the configuration in each one to use the new password.
Click Save to apply the changes.
Go back to Workloads > Pods and select the united-manufacturing-hub-hivemqce-0 Pod.
Click the Delete button to delete the Pod.

What’s next

Setting up HiveMQ PKI

5.3 - Setup PKI for the MQTT Broker

This page describes how to setup the Public Key Infrastructure (PKI) for the MQTT broker.

If you want to use MQTT over TLS (MQTTS) or Secure Web Socket (WSS) you need to setup a Public Key Infrastructure (PKI).

Read the blog article about secure communication in IoT to learn more about encryption and certificates.

Structure overview

The Public Key Infrastructure for HiveMQ consists of two Java Key Stores (JKS):

Keystore: The Keystore contains the HiveMQ certificate and private keys. This store must be confidential, since anyone with access to it could generate valid client certificates and read or send messages in your MQTT infrastructure.
Truststore: The Truststore contains all the clients public certificates. HiveMQ uses it to verify the authenticity of the connections.

Before you begin

You need to have the following tools installed:

OpenSSL. If you are using Windows, you can install it with Chocolatey.
Java

Create a Keystore

Open a terminal and run the following command:

keytool -genkey -keyalg RSA -alias hivemq -keystore hivemq.jks -storepass <password> -validity <days> -keysize 4096 -dname "CN=united-manufacturing-hub-mqtt" -ext "SAN=IP:127.0.0.1"

Replace the following placeholders:

<password>: The password for the keystore. You can use any password you want.
<days>: The number of days the certificate should be valid.

The command runs for a few minutes and generates a file named hivemq.jks in the current directory, which contains the HiveMQ certificate and private key.

If you want to explore the contents of the keystore, you can use Keystore Explorer.

Generate client certificates

Open a terminal and create a directory for the client certificates:

mkdir pki

Follow these steps for each client you want to generate a certificate for.

Create a new key pair:

openssl req -new -x509 -newkey rsa:4096 -keyout "pki/<servicename>-key.pem" -out "pki/<servicename>-cert.pem" -nodes -days <days> -subj "/CN=<servicename>"

Convert the certificate to the correct format:

openssl x509 -outform der -in "pki/<servicename>-cert.pem" -out "pki/<servicename>.crt"

Import the certificate into the Truststore:

keytool -import -file "pki/<servicename>.crt" -alias "<servicename>" -keystore hivemq-trust-store.jks -storepass <password>

Replace the following placeholders:

<servicename> with the name of the client. Use the service name from the Network > Services tab in UMHLens / OpenLens.
<days> with the number of days the certificate should be valid.
<password> with the password for the Truststore. You can use any password you want.

Import the PKI into the United Manufacturing Hub

First you need to encode in base64 the Keystore, the Truststore and all the PEM files. Use the following script to encode everything automatically:

Windows
macOS/Linux

Get-ChildItem .\ -Recurse -Include *.jks,*.pem | ForEach-Object {
    $FileContent = Get-Content $_ -Raw
    $fileContentInBytes = [System.Text.Encoding]::UTF8.GetBytes($FileContent)
    $fileContentEncoded = [System.Convert]::ToBase64String($fileContentInBytes)
    $fileContentEncoded > $_".b64"
    Write-Host $_".b64 File Encoded Successfully!"
}

find ./ -regex '.*\.jks\|.*\.pem' -exec openssl base64 -A -in {} -out {}.b64 \;

You could also do it manually with the following command:

openssl base64 -A -in <filename> -out <filename>.b64

Now you can import the PKI into the United Manufacturing Hub. To do so:

Open UMHLens / OpenLens.
Navigate to Helm > Releases.
Select the united-manufacturing-hub release.
Click the Upgrade button.
Find the _000_commonConfig.infrastructure.mqtt.tls section.
Update the value of the keystoreBase64 field with the content of the hivemq.jks.b64 file and the value of the keystorePassword field with the password you used for the keystore.
Update the value of the truststoreBase64 field with the content of the hivemq-trust-store.jks.b64 file and the value of the truststorePassword field with the password you used for the truststore.
Update the value of the <servicename>.cert field with the content of the <servicename>-cert.pem.b64 file and the value of the <servicename>.key field with the content of the <servicename>-key.pem.b64 file.
Click the Upgrade button to apply the changes.

What’s next

Learn more about HiveMQ’s TLS configuration in the HiveMQ documentation.

Production Guide

1 - Installation

1.1 - Flatcar Installation (Bare Metal)

Before you begin

Install Flatcar Linux on the edge device

Connect to the edge device

Import the cluster configuration

Access the UMH stack

Troubleshooting

The installation stops at the green login prompt

I don’t know which drive to select

I can access the cluster but there are no resources

What’s next

1.2 - Flatcar Installation (Virtual Machine)

Before you begin

Create a virtual machine

Install Flatcar Linux

Connect to the virtual machine

Import the cluster configuration

Access the UMH stack

Troubleshooting

The installation stops at the green login prompt

I can access the cluster but there are no resources

I can’t SSH into the virtual machine

What’s next

1.3 - Local k3d Installation

Before you begin

Install dependencies

Create a cluster

Install the UMH stack

Troubleshooting

I don’t see the cluster in UMHLens / OpenLens

What’s next

2 - Upgrading

2.1 - Upgrade to v0.9.14

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

2.2 - Upgrade to v0.9.13

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

2.3 - Upgrade to v0.9.12

Add Helm repo in UMHLens / OpenLens

Backup RBAC configuration for MQTT Broker

Clear Workloads

Remove MQTT Broker extension PVC

Upgrade Helm Chart

2.4 - Upgrade to v0.9.11

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

2.5 - Upgrade to v0.9.10

Add Helm repo in UMHLens / OpenLens

Grafana plugins

Clear Workloads

Upgrade Helm Chart

Replace VerneMQ with HiveMQ

Troubleshooting

Some microservices can’t connect to the new MQTT broker

ChirpStack can’t connect to the new MQTT broker

2.6 - Upgrade to v0.9.9

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

2.7 - Upgrade to v0.9.8

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

2.8 - Upgrade to v0.9.7

Add Helm repo in UMHLens / OpenLens

Clear Workloads

Upgrade Helm Chart

Change Factoryinsight API version

2.9 - Upgrade to v0.9.6

Add Helm repo in UMHLens / OpenLens

Add new index to the database

Open a shell in the database

Create the index

Clear Workloads