This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

The OSS blueprint for the Industrial IoT

The United Manufacturing Hub is an Open-Source Helm Chart for Kubernetes, which combines state-of -the-art IT / OT tools & technologies and brings them into the hands of the engineer.

Bringing the worlds best IT and OT tools into the hands of the engineer

Why start from scratch when you can leverage a proven open-source blueprint? Kafka, MQTT, Node-RED, TimescaleDB and Grafana with the press of a button - tailored for manufacturing and ready-to-go


What can you do with it?


Everything That You Need To Do To Generate Value On The Shopfloor

Prevent Vendor Lock-In and Customize to Your Needs

  • The only requirement is Kubernetes, which is available in various flavors, including k3s, bare-metal k8s, and Kubernetes-as-a-service offerings like AWS EKS or Azure AKS
  • Swap components with other options at any time. Not a fan of Node-RED? Replace it with Kepware. Prefer a different MQTT broker? Use it!
  • Leverage existing systems and add only what you need.

Get Started Immediately

Connect with Like-Minded People

  • Tap into our community of experts and ask anything. No need to depend on external consultants or system integrators.
  • Leverage community content, from tutorials and Node-RED flows to Grafana dashboards. Although not all content is enterprise-supported, starting with a working solution saves you time and resources.
  • Get honest answers in a world where many companies spend millions on advertising.

How does it work?

Only requirement: a Kubernetes cluster (and we'll even help you with that!). You only need to install the United Manufacturing Hub Helm Chart on that cluster and configure it.

The United Manufacturing Hub will then generate all the required files for Kubernetes, including auto-generated secrets, various microservices like bridges between MQTT / Kafka, datamodels and configurations. From there on, Kubernetes will take care of all the container management.



FAQ

This looks great, but the technologies mentioned here (Docker, Kubernetes, etc.) look complicated and we as a company are no experts. Can we still use it?
Yes - the United Manufacturing Hub is targeting specifically people and companies, who do not have the budget and/or knowledge to work on their own / develop everything from scratch.

With our extensive documentation, guides and knowledge sections you can learn everything that you need.

The United Manufacturing Hub abstracts these tools and technologies so that you can leverage all advantages, but still focus on what really matters: digitizing your production.

With our commercial Management Console you can manage your entire IT / OT infrastructure and work with Grafana / Node-RED without the need to ever touch or understand Kubernetes, Docker, Firewalls, Networking or similar.

Additionally, you can get support licenses providing unlimited support during development and maintenance of the system. Take a look at our website if you want to get more information on this.
Because very often these solutions do not target the actual pains of an engineer: implementation and maintenance. And then companies struggle in rolling out IIoT as the projects take much longer and cost way more than originally proposed.

In the United Manufacturing Hub, implementation and maintenance of the system are the first priority. We've had these pains too often ourselves and therefore incorporated and developed tools & technologies to avoid them.

For example, with sensorconnect we can retrofit production machines where it is impossible at the moment to extract data. Or, with our modular architecture we can fit the security needs of all IT departments - from integration into a demilitarized zone to on-premise and private cloud. With Apache Kafka we solve the pain of corrupted or missing messages when scaling out the system

How to proceed?

1 - Get Started!

You want to get started right away? Go ahead and jump into the action!

Great to see you’re ready to start! This guide has 5 steps: Installation, Managing the System, Data Acquisition & Manipulation, and Moving to Production.

Contact Us!

Do you still have questions on how to get started? Message us on our Discord Server.

1.1 - 1. Installation

Install the United Manufacturing Hub together with all required tools on a Linux Operating System.

The United Manufacturing Hub (UMH) can be deployed on various external devices, including edge devices and virtual machines (VMs). For initial installations or for development purposes, it is recommended to use a VM.

Software Requirements

The UMH installation requires one of the following Operaing System on your server:

  • Flatcar version current-2023 or higher (3510.3.1). It is recommended that you have full control over the operating system. To install Flatcar on your server, follow this guide.
  • Red Hat Enterprise Linux (RHEL) 9.0 and higher. Recommended when you can choose only out of a small amount of potential Operating System in your large enterprise

While UMH is optimized for RHEL and Flatcar, it can theoretically run on other Linux distributions. However, support is not guaranteed. For Windows, you could try running one of the above described Operating Systems in a VM (e.g., Hyper-V). If you experiment with other systems, we encourage sharing your experiences on our Discord channel.

Hardware Requirements

  • CPU: Minimum 4 cores
  • Memory: 16 GB RAM
  • Disk Space: 32 GB available

Note: Systems at the edge of these requirements may experience longer installation times. Close other programs during installation for optimal performance.

Installation Steps

  1. Open the Management Console in the browser.

  2. When you are finished with the creation of your account, enter your information and click on SIGN IN.

    Untitled
    Untitled

  3. If you are not a member, continue with sign up. Register your information and click on SIGN UP.

    Untitled
    Untitled

  4. Click on +ADD INSTANCE button.

    Untitled
    Untitled

  5. Select Install UMH Only.

    Untitled
    Untitled

  6. Enter your instace name and then click on CREATE MY COMMAND.

    Untitled
    Untitled

  7. You should be able to see a create command. Copy and paste the following command into your server’s terminal (via ssh).

    Untitled
    Untitled

  8. The installation script runs a lot of checking and setup. For example, it checks your operating system, installation of required tools, and internet connection. After the check phase, kubectl and Helm will be installed. The script shall show you what actions will happen to your system in the next step. If you want to proceed, type Y and press enter key.

    Untitled
    Untitled

  9. In this step, k3s will be installed. Then, it installs the UMH Helm Chart in Kubernetes. After that, the Management Companion will be installed into Kubernetes. Until everything is set up, it can take a while.

    Untitled
    Untitled

  10. After successful installation, you should be able to see messages like in the picture below.

    Untitled
    Untitled

  11. Go back to the Management Console and click on LET’S GO!

    Untitled
    Untitled

  12. Now, you should be able to see your instance on the dashboard.

    Untitled
    Untitled

Do you need more technical background information?

Here are some links to get you started:

What’s next?

Once you installed UMH, you can continue with the next page to learn how to manage the system, for example, access to the microservices.

1.2 - 2. Managing the System

Learn how to manage your UMH instance with the Management Console.

In this chapter, you will learn how to monitor, manage and configure your UMH instance with the Management Console.

At this stage, you should have already installed the UMH on your device. If you have not done so, please follow the steps in the Installation chapter first.

A Few Words About the Communication

Now that you have connected a UMH instance to the Management Console, you might be curious about how the Management Console communicates with the instance.

The Management Companion, serving as an agent within each UMH instance, provides a secure link to the Management Console. It enables comprehensive and secure monitoring and management of the UMH, ensuring system health and streamlined configuration, all while acting as a vigilant watchdog over system components and connected devices.

The diagram below illustrates the communication flow between the Management Console and the instance:

Communication between the Management Console and the instance
Communication between the Management Console and the instance

For more information, visit the architecture

Overview of Your Instances

Instance overview
Instance overview

On the left side of the Management Console, you can view the list of your instances. If you have just installed the UMH, you should see only one instance in the list.

The status of your instance is indicated by color: green means everything is working properly, while yellow indicates that there may be a connection issue.

The Messages Received statistic shows the number of messages received by you from the instance since you opened the Management Console. It is usually a good indicator of the health of the connection to the Companion. If the number is not increasing for 10 seconds, the instance is considered disconnected.

Monitoring the Instance’s Status

From the instance dashboard, in the overview tab, you can view the status of your instance. There are multiple interfaces that display the status of each component of the system.

Modules

A Module refers to a group of workloads in the United Manufacturing Hub responsible for specific tasks. For example, the Historian & Analytics module represents the microservices, storage, and connections that are responsible for storing and analyzing data.

In the Modules tab, you can view the status of each module. If a module is not healthy, it means that one or more of its components are not functioning properly.

System

In the System tab, you can view the resource usage of your device, as well as some system’s information.

If there is an overload on the device, you can view it here. An overloaded device is unable to handle the workload, and you should consider upgrading the device.

Data

In the Data tab, you can see an overview of the data infrastructure, including the number of messages going through the message broker, those stored in the database, and messages received by each data source.

Connection

In the Connection tab, you can view the status of all the data source connections configured in the system. A non-healthy connection indicates that the device and the data source are unable to communicate.

Kubernetes

In the Kubernetes tab, you can check for any error events in the Kubernetes cluster. Any errors suggest that the cluster is not operating correctly.

Additionally, this tab displays the version of the United Manufacturing Hub and the Management Companion currently installed on your device.

Manage the Instance

Before you begin, ensure that you are connected to the same network as the instance for accessing the various services and features discussed below.

While a graphical user interface for managing the instance is not yet available, you can still manage it via the command line.

Access the Command Line

Access your device’s shell either directly or via SSH. Note: Root user access is required for the following commands.

In UMH’s current version, add --kubeconfig /etc/rancher/k3s/k3s.yaml to each kubectl command. Root privileges are needed to access it. The installation path of kubectl might vary (e.g., /usr/local/bin/kubectl on RHEL/Linux, /opt/bin/kubectl on flatcar). These paths may not be in the root user’s PATH, so the commands below might appear complex.

Interact with the Instance

First, set this environment variable:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

You can bypass this by adding –kubeconfig /etc/rancher/k3s/k3s.yaml to your commands. All instructions in this chapter will include this flag.

Then, to get a list of pods, run:

sudo $(which kubectl) get pods -n united-manufacturing-hub  --kubeconfig /etc/rancher/k3s/k3s.yaml

For a comprehensive list of commands, refer to the Kubernetes documentation.

Always specify the namespace when running a command by adding -n united-manufacturing-hub.

Access Node-RED

Node-RED is used in UMH for creating data flows. Access it via:

http://<instance-ip-address>:1880/nodered

Access Grafana

UMH uses Grafana for dashboard displays. Get your credentials:

sudo $(which kubectl) get secret grafana-secret --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub -o jsonpath="{.data.adminuser}" | base64 --decode; echo
sudo $(which kubectl) get secret grafana-secret --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub -o jsonpath="{.data.adminpassword}" | base64 --decode; echo

Then, access Grafana here:

http://<instance-ip-address>:8080

Use the retrieved credentials to log in.

Access the RedPanda Console

Manage the Kafka broker via the RedPanda Console:

http://<instance-ip-address>:8090

Interact with the Database

UMH uses TimescaleDB. Open a psql session:

sudo $(which kubectl) exec -it $(sudo $(which kubectl) get pods --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub -l app.kubernetes.io/component=timescaledb -o jsonpath="{.items[0].metadata.name}") --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub -- psql -U postgres

Run SQL queries as needed. For an overview of the database schema, refer to the Data Model documentation.

Connect MQTT to MQTT Explorer

Use MQTT Explorer for a structured overview of MQTT topics. Connect using the instance’s IP and port 1883.

Troubleshooting

Error: You must be logged in to the server while using the kubectl Command

If you encounter the error below while using the kubectl command:

E1121 13:05:52.772843  218533 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
error: You must be logged in to the server (the server has asked for the client to provide credentials)

This issue can be resolved by setting the KUBECONFIG environment variable. Run the following command:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Alternatively, use the --kubeconfig flag to specify the configuration file path:

sudo $(which kubectl) --kubeconfig /etc/rancher/k3s/k3s.yaml get pods -n united-manufacturing-hub

“Permission Denied” Error with kubectl Command

Encountering the error below while using the kubectl command:

error: error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied

Indicates the need for root access. Run the command with sudo, or log in as the root user.

kubectl: command not found error

If you encounter the error below while using the kubectl command:

kubectl: command not found

The solution is to use the full path to the kubectl binary. You can do this by prefixing the command with /usr/local/bin/ (for RHEL and other Linux systems), or /opt/bin/ (for flatcar) or by adding it to your PATH environment variable:

/usr/local/bin/kubectl get pods -n united-manufacturing-hub

# or

export PATH=$PATH:/usr/local/bin

Viewing Pod Logs for Troubleshooting

Logs are essential for diagnosing and understanding the behavior of your applications and infrastructure. Here’s how to view logs for key components:

  • Management Companion Logs: To view the real-time logs of the Management Companion, use the following command. This can be helpful for monitoring the Companion’s activities or troubleshooting issues.

    sudo $(which kubectl) logs -f mgmtcompanion-0 -n mgmtcompanion --kubeconfig /etc/rancher/k3s/k3s.yaml

  • TimescaleDB Logs: For real-time logging of the TimescaleDB, execute this command. It’s useful for tracking database operations and identifying potential issues.

    sudo $(which kubectl) logs -f united-manufacturing-hub-timescaledb-0 -n united-manufacturing-hub --kubeconfig /etc/rancher/k3s/k3s.yaml

Restarting a Pod for Troubleshooting

Sometimes, the most straightforward troubleshooting method is to restart a problematic pod. Here’s how to restart specific pods:

  • Restart Management Companion: If you encounter issues with the Management Companion, restart it with this command:

    sudo $(which kubectl) delete pod mgmtcompanion-0 -n mgmtcompanion --kubeconfig /etc/rancher/k3s/k3s.yaml

  • Restart TimescaleDB: Should TimescaleDB exhibit unexpected behavior, use the following command to restart it:

    sudo $(which kubectl) delete pod united-manufacturing-hub-timescaledb-0 -n united-manufacturing-hub --kubeconfig /etc/rancher/k3s/k3s.yaml

Troubleshooting Redpanda / Kafka

For insights into your Kafka streams managed by Redpanda, these commands are invaluable:

  • List All Topics: To get an overview of all topics in your Redpanda cluster:

    sudo $(which kubectl) exec -it --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub  united-manufacturing-hub-kafka-0 -- rpk topic list

  • Describe a Specific Topic: For detailed information about a specific topic, such as umh.v1.e2e-enterprise.aachen.packaging, use:

    sudo $(which kubectl) exec -it --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub united-manufacturing-hub-kafka-0 -- rpk topic describe umh.v1.e2e-enterprise.aachen.packaging

  • Consume Messages from a Topic: To view messages from a topic like umh.v1.e2e-enterprise.aachen.packaging, this command is useful for real-time data observation:

    sudo $(which kubectl) exec -it --kubeconfig /etc/rancher/k3s/k3s.yaml -n united-manufacturing-hub united-manufacturing-hub-kafka-0 -- rpk topic consume umh.v1.e2e-enterprise.aachen.packaging

What’s next?

Now that you have learned how to monitor, manage and configure your UMH instance with the Management Console, you can start creating your first data flow. To learn how to do this, proceed to the Data Acquisition and Manipulation chapter.

1.3 - 3. Data Acquisition and Manipulation

Learn how to connect various data sources to the UMH and format data into the UMH data model.

The United Manufacturing Hub excels in its ability to integrate diverse data sources and standardize data into a unified model, enabling seamless integration of existing data infrastructure for analysis and processing.

Currently, data sources can be connected to the UMH through Benthos for OPC UA and Node-RED for other types.

The UMH includes 3 pre-configured data simulators for testing connections:

Connect OPC UA Data Sources

OPC UA, often complex, can be streamlined using our Benthos-based OPC UA connector accessible from the Management Console.

Create a Connection with the Management Console

After logging into the Management Console and selecting your instance, navigate to the Data Connections tab to view existing connections.

Data Connections Overview
Data Connections Overview

Uninitialized Connections are established but not yet configured as data sources, while Initialized Connections are fully configured.***h

The health status reflects the UMH-data source connection, not data transmission status.

To add a new connection, click Add Connection. Currently, the only option is OPC UA Server. Enter the server details, including the unique name and address with protocol (opc.tcp://) and port (usually 4840).

For testing with the OPC UA simulator, use:

opc.tcp://united-manufacturing-hub-opcuasimulator-service:46010

OPC UA Connection Details
OPC UA Connection Details

Test the connection, and if successful, click Add Connection.

Initialize the Connection

After adding, you must initialize the connection. This creates a new Benthos deployment for data publishing to the UMH Kafka broker.

Navigate to Data Sources > Uninitialized Connections and initiate the connection.

Initialize Connection
Initialize Connection

Enter authentication details (use Anonymous for no authentication, as with the OPC UA simulator).

Specify OPC UA nodes to subscribe to in a yaml file, following the ISA95 standard:

  nodes:
    - opcuaID: ns=2;s=Pressure
      enterprise: pharma-genix
      site: aachen
      area: packaging
      line: packaging_1
      workcell: blister
      originID: PLC13
      tagName: machineState
      useCase: _historian

Mandatory fields are opcuaID, enterprise, tagName and useCase.

Learn more about Data Modeling in the Unified Namespace in the Learning Hub.

Review and confirm the nodes, then proceed with initialization. Successful initialization will be indicated by a green message.

The new data source will now appear in the Data Sources section.

Data Sources Overview
Data Sources Overview

Connect MQTT Servers

There are a lot of options to connect an MQTT server to the UMH. For this guide, we’ll use Node-RED to connect to the MQTT simulator and format data into the UMH data model.

To access Node-RED’s web interface, navigate to:

http://<instance-ip-address>:1880/nodered

Replace <instance-ip-address> with your UMH instance’s IP. Ensure you’re on the same network for access.

Add the MQTT Connection

In Node-RED, find the mqtt-in node from the node palette and drag it into your flow. Double-click to configure and click the pencil button next to the Server field.

Enter your MQTT broker’s details:

  • Server: united-manufacturing-hub-mqtt
  • Port: 1883

For the purpose of this guide, we’ll use the UMH MQTT broker, even though the data coming from it is already bridged to Kafka by the MQTT Kafka Bridge. Since the simulated data is using the old Data Model, we’ll use Node-RED to convert it to the new Data Model.

Click Add to save.

Connect MQTT to Node-RED
Connect MQTT to Node-RED

Define the subscription topic. For example, ia/raw/development/ioTSensors/Temperature is used by the MQTT Simulator.

To test, link a debug node to the mqtt-in node and deploy. Open the debug pane by clicking on the bug icon on the top right of the screen to view messages from the broker.

MQTT Debug Connection
MQTT Debug Connection

Explore Unified Namespace for details on topic structuring.

Format Incoming Messages

Use a function node to format raw data. Connect it to the mqtt-in node and paste this script:

msg.payload = {
    "timestamp_ms": Date.now(),
    "temperature": msg.payload
}
return msg;

Finalize with Done.

Then, connect a JSON node to the function node to parse the object into a string.

This function transforms the payload into the correct format for the UMH data model.

Send Formatted Data to Kafka

For this guide, we’ll send data to the UMH Kafka broker.

Ensure you have node-red-contrib-kafkajs installed. If not, see How to Get Missing Plugins in Node-RED.

Add a kafka-producer node, connecting it to the JSON node. Configure as follows:

  • Brokers: united-manufacturing-hub-kafka:9092
  • Client ID: nodered

Update to save.

Structure Kafka topics according to UMH data model:

umh.v1.<enterprise>.<site>.<area>.<line>.<workcell>.<originID>.<useCase>.<tagName>

Example topic for this tutorial:

umh.v1.pharma-genix.aachen.packaging.packaging_1.blister.PLC13._historian.temperatureCelsius

To learn more about the UMH data-model, read the documentation.

Click Done and deploy.

Optional: Add a debug node for output visualization.

Node-RED MQTT to Kafka
Node-RED MQTT to Kafka

Connect Kafka Data Sources

Kafka data sources can be integrated with UMH exclusively through Node-RED.

To access Node-RED’s web interface, navigate to:

http://<instance-ip-address>:1880/nodered

Replace <instance-ip-address> with your UMH instance’s IP, ensuring you’re on the same network for access.

Before proceeding, make sure the node-red-contrib-kafkajs plugin is installed. For installation guidance, see How to Get Missing Plugins in Node-RED.

Add the Kafka Connection

In Node-RED, locate the kafka-consumer node and drag it into your flow. Double-click to configure and click the pencil button beside the Server field.

If you have followed the guide, the kafka client should already be configured and automatically selected.

Enter your Kafka broker’s details:

  • Brokers: united-manufacturing-hub-kafka:9092
  • Client ID: nodered

Click Add to save.

Connect Kafka to Node-RED
Connect Kafka to Node-RED

Set the subscription topic. For demonstration, we’ll use the topic created earlier:

umh.v1.pharma-genix.aachen.packaging.packaging_1.blister.PLC13._historian.temperatureCelsius

Link a debug node to the kafka-consumer node, deploy, and observe messages in the debug pane.

Kafka Debug Connection
Kafka Debug Connection

For topic structuring guidelines, refer to Unified Namespace.

Format Incoming Messages

Since the data is already processed from the previous step, use a function node to convert the temperature from Celsius to Fahrenheit. Connect it to the kafka-consumer node and paste the following script:

const payloadObj = JSON.parse(msg.payload.value)
const celsius = payloadObj.temperature
const fahrenheit = (celsius * 9 / 5) + 32

msg.payload = {
    "timestamp_ms": Date.now(),
    "temperature": fahrenheit
}

return msg;

Finalize with Done.

Then, connect a JSON node to the function node to parse the object into a string.

Send Formatted Data Back to Kafka

Now, we’ll route the transformed data back to the Kafka broker, in a different topic.

Add a kafka-producer node, connecting it to the JSON node. Use the same Kafka client as earlier, and the same topic for output:

umh.v1.pharma-genix.aachen.packaging.packaging_1.blister.PLC13._historian.temperatureFahrenheit

For more on UMH data modeling, consult the documentation.

Press Done and deploy.

Consider adding a debug node for visualizing output data.

Node-RED Kafka to Kafka
Node-RED Kafka to Kafka

What’s next

Next, we’ll dive into Data Visualization, where you’ll learn to create Grafana dashboards using your newly configured data sources. This next chapter will help you visualize and interpret your data effectively.

1.4 - 4. Data Visualization

Build a simple Grafana dashboard with the gathered data.

In the following step, we will delve into the process of visualizing the data. This chapter focuses on the construction of dashboards using Grafana. The dashboard will be crafted around the OPC-UA data source and the Node-RED flow, both of which were established in the previous chapter.

Creating a Grafana dashboard

  1. If you haven’t done so already, open and log in to Grafana by following the instructions given in the Acess Grafana section of chapter 2.

  2. Once logged in, hover over the fourth icon in the left menu, dashboards, and click on + New dashboard.

    Untitled
    Untitled

  3. Click on Add a new panel, which will redirect you to the edit panel view.

  4. Next, we’ll retrieve OPC-UA data from TimescaleDB. Before moving forward, ensure that the UMH TimescaleDB data source is selected; it should be the default choice.

    Untitled
    Untitled

  5. We’ll show you how to run queries with both the Builder mode (a graphical query builder) and the Code mode (a code editor to write RAW SQL). Let’s begin with the graphical approach.

  6. Let’s query all value, timestamp and name columns from the tag table. For some guidance, refer to the image below.

    Untitled
    Untitled

  7. Click on the Run Query button located next to the Builder/Code modes switcher.

  8. You should now see a time series graph based on the query you just ran. The Builder mode is a great way to get started, but it has its limitations. For more complex use cases, we recommend using the Code mode, which we’ll cover in the next steps.

  9. Open the code editor by switching from Builder to Code.

  10. Now we’ll run a slightly more complex query. We’ll retrieve the same columns as before, but this time only for a specific asset.

    SELECT name, value, timestamp
FROM tag
WHERE asset_id = get_asset_id(
  'pharma-genix',
  'aachen',
  'packaging',
  'packaging_1',
  'blister'
);

get_asset_id is a custom plpgsql function that we provide to simplify the process of querying tag data from a specific asset. In the code snippet above, the arguments provided to the function are based on the OPC-UA node we defined in the Initialize the Connection section of the previous chapter, so adjust them accordingly if you used different values, these values can also be found at the data dashboard of the Management Console, which displays all OPC-UA nodes in a tree structure.

  1. Same as before, click on the Run Query button to execute the query. If you’ve been following along, you won’t see any noticeable changes, since we only have one asset in our database.

  2. Feel free to experiment with different queries to get a better feel for the data model.

  3. Next, you can customize your dashboard. On the right side, you’ll find various options, such as specifying units or setting thresholds. Playaround until it suits your needs.

  4. Once you’re done making adjustments, click on the blue Apply button in the top right-hand corner to save the panel and return to the overview.

  5. Congratulations, you have created your first Grafana dashboard, and for now it should look similar to the one below.

    Untitled
    Untitled

What’s next?

The next topic is Moving to Production where we will explain what it means to move the umh to a manufacturing environment. Click here to proceed.

1.5 - 5. Moving to Production

Move the United Manufacturing Hub to production.

This chapter involves deploying the United Manufacturing Hub (UMH) on a virtual machine or an edge device, allowing you to connect with your production assets. However, we recognize the importance of familiarizing yourself with the United Manufacturing Hub beforehand. Feel free to delve deeper into our product, explore the specifics of local installation, or proceed with the production deployment. The guide below will kickstart your UMH journey in a production setting.

Check out our community

We are quite active on GitHub and Discord.

Feel free to join our community, introduce yourself and share your best-practices and experiences.

Learn more about the United Manufacturing Hub

If you like reading more about its features and architecture, check out the following chapters:

  • Features to understand the capabilities of the United Manufacturing Hub and learn how to use them.
  • Architecture to learn what is behind the United Manufacturing Hub and how everything works together.

Ready to transition to production? Continue reading to discover how to install UMH and seamlessly connect multiple machines to your instance.

Set up your first instance and connect to a few machines

If you want to get a first impression of the UMH in a production environment, connecting to machines on your shop floor, follow these steps:

Before starting the installation process, decide whether to use virtual machine (VM), or a generic server or edge device. For ease of setup, we recommend using a VM. Ensure that the selected device has network access to the machines.

2. Select Machines with OPC UA for Testing

For testing purposes, it’s recommended to use machines with OPC UA. If your machines use other protocols, consider Node-RED as an alternative for data connection. Check the list of supported protocols and how to connect them to Node-RED.

3. Installation Process

The installation is well documented in the first chapter. But here’s a quick overview:

  • Click on the + Add Instance button in the instance dashboard of the Management Console, redirecting you to the installation process page.

  • Select the Install UMH Only option, redirecting you to the install command generation page.

  • Finally, follow the provided instructions to set up your instance. If everything went well, there should be a button at the bottom right corner of the page, redirecting you back to the instance dashboard.

4. Network Configuration

Once your UMH instance is up and running, ensure it is placed in the same network as your machines. Additionally, verify that the device running the Management Console is also within the same network.

While basic management and monitoring with the Management Console don’t necessarily demand extensive network configuration, it’s important to note that various open-source tools integrated into UMH do. Therefore, to take full advantage of the UMH, ensure that you can reach the IP of the server to access Grafana or Node-RED, and that the connection is not blocked by a firewall.

5. Configure Connections and Data Sources

The connections and data sources setup is documented in detail in the third chapter. But here’s a quick overview for OPC UA:

  • Assuming you have a selected instance running, navigate to the Data Connections tab and click on the + Add Connection button.
  • Click on the OPC UA Server option, redirecting you to the OPC UA connection setup page.
  • Follow the provided instructions, test the connection, and if succesful you’ll be able to deploy it.
  • Once successfully deployed and back to the Data Connections tab, you’ll see the new connection under Uninitialized Connections.
  • To initialize it, navigate to Data Sources > Uninitialized Connections, redirecting you to the data source setup page.
  • Again, follow the provided instructions. If you’re having trouble, refer to the more detailed guide.
  • Once successfully deployed and back to the Data Sources tab, you’ll see it under Data Sources.
  • Congratulations! Your OPC UA connection and data source are now configured, and your UMH instance is ready to gather valuable insights from your machines.

6. Alternative Protocols

The previous step exclusively covered OPC UA. If your machines use protocols other than OPC UA, we recommend exploring Node-RED as a versatile solution for connecting and gathering data.

You can use it to generate a new dataflow by using your machine’s data as input. This collected data can even be used in other tools, such as building a dashboard in Grafana.

If you encounter any issues, feel free to ask for help on our Discord channel.

Play around with it locally

If you want to get a first impression of the UMH in a local environment, we recommend checking out the following topics:

Grafana Canvas

If you’re interested in creating visually appealing Grafana dashboards, you might want to try Grafana-Canvas. In our previous blog article, we explained why Grafana-Canvas is a valuable addition to your standard Grafana dashboard. If you’d like to learn how to build one, check out our tutorial.

Untitled
Untitled

OPC/UA-Simulator

If you want to get a good overview of how the OPC/UA protocol works and how to connect it to the UMH, the OPC/UA-simulator is a useful tool. Detailed instructions can be found in this guide.

Untitled
Untitled

PackML-Simulator

For those looking to get started with PackML, the PackML Simulator is another helpful simulator. Check out our tutorial on how to create a Node-RED flow with PackML data.

Untitled
Untitled

Benthos

Benthos is a highly scalable data manipulation and IT connection tool. If you’re interested in learning more about it, check out our tutorial.

Untitled
Untitled

Kepware

At times, you may need to connect different, older protocols. In such cases, KepwareServerEx can help bridge the gap between these older protocols and the UMH. If you’re interested in learning more, check out our tutorial.

Deployment to production

Ready to go to production? Go install it!

Follow our step-by-step tutorial on how to install the UMH on an edge device or an virtual machine using Flatcar. We’ve also written a blog article explaining why we use Flatcar as the operating system for the industrial IoT, which you can find here.

Make sure to check out our advanced production guides, which include detailed instructions on how to secure your setup and how to best integrate with your infrastructure.

2 - Features

Do you want to understand the capabilities of the United Manufacturing Hub, but do not want to get lost in technical architecture diagrams? Here you can find all the features explained on few pages.

2.1 - Connectivity

This page describes the connectivity features of the United Manufacturing Hub.

2.1.1 - Node-RED

Connect devices on the shop floor using Node-RED with United Manufacturing Hub’s Unified Namespace. Simplify data integration across PLCs, Quality Stations, and MES/ERP systems with a user-friendly UI.

One feature of the United Manufacturing Hub is to connect devices on the shopfloor such as PLCs, Quality Stations or MES / ERP systems with the Unified Namespace using Node-RED. Node-RED has a large library of nodes, which lets you connect various protocols. It also has a user-friendly UI with little code, making it easy to configure the desired nodes.

When should I use it?

Sometimes it is necessary to connect a lot of different protocols (e.g Siemens-S7, OPC-UA, Serial, …) and node-RED can be a maintainable solution to connect all these protocols without the need for other data connectivity tools. Node-RED is largely known in the IT/OT-Community making it a familiar tool for a lot of users.

What can I do with it?

By default, there are connector nodes for common protocols:

  • connect to MQTT using the MQTT node
  • connect to HTTP using the HTTP node
  • connect to TCP using the TCP node
  • connect to IP using the UDP node

Furthermore, you can install packages to support more connection protocols. For example:

You can additionally contextualize the data, using function or other different nodes do manipulate the received data.

How can I use it?

Node-RED comes preinstalled as a microservice with the United Manufacturing Hub.

  1. To access Node-RED, simply open the following URL in your browser:
http://<instance-ip-address>:1880/nodered

  1. Begin exploring right away! If you require inspiration on where to start, we provide a variety of guides to help you become familiar with various node-red workflows, including how to process data and align it with the UMH datamodel:

What are the limitations?

  • Most packages have no enterprise support. If you encounter any errors, you need to ask the community. However, we found that these packages are often more stable than the commercial ones out there, as they have been battle tested by way more users than commercial software.
  • Having many flows without following a strict structure, leads in general to confusion.
  • One additional limitation is “the speed of development of Node-RED”. After a big Node-RED and JavaScript update dependencies most likely break, and those single community maintained nodes need to be updated.

Where to get more information?

2.1.2 - Additional Connectivity

2.1.2.1 - Retrofitting with ifm IO-link master and sensorconnect

Upgrade older machines with ifm IO-Link master and Sensorconnect for seamless data collection and integration. Retrofit your shop floor with plug-and-play sensors for valuable insights and improved efficiency.

Retrofitting older machines with sensors is sometimes the only-way to capture process-relevant information. In this article, we will focus on retrofitting with ifm IO-Link master and Sensorconnect, a microservice of the United Manufacturing Hub, that finds and reads out ifm IO-Link masters in the network and pushes sensor data to MQTT/Kafka for further processing.

When should I use it?

Retrofitting with ifm IO-Link master such as the AL1350 and using Sensorconnect is ideal when dealing with older machines that are not equipped with any connectable hardware to read relevant information out of the machine itself. By placing sensors on the machine and connecting them with IO-Link master, required information can be gathered for valuable insights. Sensorconnect helps to easily connect to all sensors correctly and properly capture the large amount of sensor data provided.

What can I do with it?

With ifm IO-Link master and Sensorconnect, you can collect data from sensors and make it accessible for further use. Sensorconnect offers:

  • Automatic detection of ifm IO-Link masters in the network.
  • Identification of IO-Link and alternative digital or analog sensors connected to the master using converter such as the DP2200. Digital Sensors employ a voltage range from 10 to 30V DC, producing binary outputs of true or false. In contrast, analog sensors operate at 24V DC, with a current range spanning from 4 to 20 mA. Utilizing the appropriate converter, analog outputs can be effectively transformed into digital signals.
  • Constant polling of data from the detected sensors.
  • Interpreting the received data based on a sensor database containing thousands of entries.
  • Sending data in JSON format to MQTT and Kafka for further data processing.

How can I use it?

To use ifm IO-link gateways and Sensorconnect please follow these instructions:

  1. Ensure all IO-Link gateways are in the same network or accessible from your instance of the United Manufacturing Hub.
  2. Retrofit the machines by connecting the desired sensors and establish a connection with ifm IO-Link gateways.
  3. Configure the Sensorconnect IP-range to either match the IP address using subnet notation /32, or, in cases involving multiple masters, configure it to scan an entire range, for example /24. To change the value, go to the Customize the United Manufacturing Hub section.
  4. Once completed, the data should be available in your Unified Namespace.

What are the limitations?

  • The current ifm firmware has a software bug, that will cause the IO-Link master to crash if it receives to many requests. To resolve this issue, you can either request an experimental firmware, which is available exclusively from ifm, or re-connect the power to the IO-Link gateway.

Where to get more information?

2.1.2.2 - Retrofitting with USB barcodereader

Integrate USB barcode scanners with United Manufacturing Hub’s barcodereader microservice for seamless data publishing to Unified Namespace. Ideal for inventory, order processing, and quality testing stations.

The barcodereader microservice enables the processing of barcodes from USB-linked scanner devices, subsequently publishing the acquired data to the Unified Namespace.

When should I use it?

When you need to connect a barcode reader or any other USB devices acting as a keyboard (HID). These cases could be to scan an order at the production machine from the accompanying order sheet. Or To scan material for inventory and track and trace.

What can I do with it?

You can connect USB devices acting as a keyboard to the Unified Namespace. It will record all inputs and send it out once a return / enter character has been detected. A lof of barcode scanners work that way. Additionally, you can also connect something like a quality testing station (we once connected a Mitutoyo quality testing station).

How can I use it?

To use the microservice barcode reader, you will need configure the helm-chart and enable it.

  1. Enable _000_commonConfig.datasources.barcodereader.enabled in the Helm Chart
  2. During startup, it will show all connected USB devices. Remember yours and then change the INPUT_DEVICE_NAME and INPUT_DEVICE_PATH
  3. Also set ASSET_ID, CUSTOMER_ID, etc. as this will then send it into the topic ia/ASSET_ID/…/barcode
  4. Restart the pod
  5. Scan a device, and it will be written into the topic xxx

Once installed, you can configure the microservice by setting the needed environment variables. The program will continuously scan for barcodes using the device and publish the data to the Kafka topic.

What are the limitations?

  • Sometimes special characters are not parsed correctly. They need to be adjusted afterward in th Unified Namespace.

Where to get more information?

2.2 - Data Infrastructure

This page describes the data infrastructure of the United Manufacturing Hub.

2.2.1 - Unified Namespace

Exchange events and messages across all your shopfloor equipment, IT / OT systems such as ERP or MES and microservices.

The Unified Namespace is an event-driven architecture that allows for seamless communication between nodes in a network. It operates on the principle that all data, regardless of whether there is an immediate consumer, should be published and made available for consumption. This means that any node in the network can work as either a producer or a consumer, depending on the needs of the system at any given time.

To use any functionalities of the United Manufacturing Hub, you need to use the Unified Namespace as well. More information can be found in our Learning Hub on the topic of Unified Namespace.

When should I use it?

An application consists always out of multiple building blocks. To connect those building blocks, one can either exchange data between them through databases, through service calls (such as REST), or through a message broker.

Opinion: We think for most applications in manufacturing, communication via a message broker is the best choice as it prevents spaghetti diagrams and allows for real-time data processing. For more information about this, you can check out this blog article.

In the United Manufacturing Hub, each single piece of information / “message” / “event” is sent through a message broker, which is also called the Unified Namespace.

What can I do with it?

The Unified Namespace / Message Broker in the United Manufacturing Hub provides several notable functionalities in addition to the features already mentioned:

  • Easy integration using MQTT: Many modern shopfloor equipment can send and receive data using the MQTT protocol.
  • Easy integration with legacy equipment: Using tools like Node-RED, data can be easily extracted from various protocols such as Siemens S7, OPC-UA, or Modbus
  • Get notified in real-time via MQTT: The Unified Namespace allows you to receive real-time notifications via MQTT when new messages are published. This can be useful for applications that require near real-time processing of data, such as an AGV waiting for new commands.
  • Retrieve past messages from Kafka logs: By looking into the Kafka logs, you can always be aware of the last messages that have been sent to a topic. This allows you to replay certain scenarios for troubleshooting or testing purposes.
  • Efficiently process messages from millions of devices: The Unified Namespace is designed to handle messages from millions of devices in your factory, even over unreliable connections. By using Kafka, you can efficiently at-least-once process each message, ensuring that each message arrives at-least-once (1 or more times).
  • Trace messages through the system: The Unified Namespace provides tracing capabilities, allowing you to understand where messages are coming from and where they go. This can be useful for debugging and troubleshooting purposes. You can use the Management Console to visualize the flow of messages through the system.to visualize the flow of messages through the system.

How can I use it?

Using the Unified Namespace is quite simple:

Configure your IoT devices and devices on the shopfloor to use the in-built MQTT broker of the United Manufacturing Hub by specifying the MQTT protocol, selecting unencrypted (1883) / encrypted (8883) ports depending on your configuration, and send the messages into a topic starting with ia/raw. From there on, you can start processing the messages in Node-RED by reading in the messages again via MQTT or Kafka, adjusting the payload or the topic to match the UMH datamodel and sending it back again to MQTT or Kafka.

If you send the messages into other topics, some features might not work correctly (see also limitations).

Recommendation: Send messages from IoT devices via MQTT and then work in Kafka only.

What are the limitations?

  • Messages are only bridged between MQTT and Kafka if they fulfill the following requirements:
    • payload is a valid JSON OR message is sent to the ia/raw topic
    • only sent to topics matching the allowed topics in the UMH datamodel, independent of what is configured in the environment variables (will be changed soon)
    • The topic lengths can be maximum 249 characters as this is a Kafka limitation
    • Only the following characters are allowed in the topic: a-z, A-Z, _ and -
    • Max. messages size for the mqtt-kafka-bridge is 0.95MB (1000000 bytes). If you have more, we recommend using Kafka directly and not bridging it via MQTT.
  • Messages from MQTT to Kafka will be published under a different topic:
    • Spaces will be removed
    • / characters will be replaced with a .
    • and vice versa
  • By default, there will be no Authorization and Authentication on the MQTT broker. You need to enable authentication and authorization yourself.
  • The MQTT or Kafka broker is not exposed externally by default. You need to enable external MQTT access first, or alternatively expose Kafka externally.

Where to get more information?

2.2.2 - Historian / Data Storage

Learn how the United Manufacturing Hub’s Historian feature provides reliable data storage and analysis for your manufacturing data.

The Historian / Data Storage feature in the United Manufacturing Hub provides reliable data storage and analysis for your manufacturing data. Essentially, a Historian is just another term for a data storage system, designed specifically for time-series data in manufacturing.

When should I use it?

If you want to reliably store data from your shop floor that is not designed to fulfill any legal purposes, such as GxP, then the United Manufacturing Hub’s Historian feature is ideal. Open-Source databases such as TimescaleDB are superior to traditional historians in terms of reliability, scalability and maintainability, but can be challenging to use for the OT engineer. The United Manufacturing Hub fills this usability gap, allowing OT engineers to easily ingest, process, and store data permanently in an Open-Source database.

What can I do with it?

The Historian / Data Storage feature of the United Manufacturing Hub allows you to:

Store and analyze data

  • Automatically store data from the processValue topics in the Unified Namespace. Data can be sent to the Unified Namespace from various sources, allowing you to store tags from your PLC and production lines reliably.
  • Conduct basic data analysis, including automatic downsampling, gap filling, and statistical functions such as Min, Max, and Avg

Query and visualize data

  • Query data in an ISA95 model, from enterprise to site, area, production line, and work cell.
  • Visualize your data in Grafana to easily monitor and troubleshoot your production processes.

More information about the exact analytics functionalities can be found in the umh-datasource-v2 documentation. Further below some screenshots of said datasource.

Efficiently manage data

  • Compress and retain data to reduce database size using various techniques.

How can I use it?

Convert your data in your Unified Namespace to processValue messages, and the Historian feature will store them automatically. You can then view the data in Grafana. An example can be found in the Getting Started guide.

For more information about what exactly is behind the Historian feature, check out our our architecture page

What are the limitations?

Apart from these limitations, the United Manufacturing Hub’s Historian feature is highly performant compared to legacy Historians.

Where to get more information?

2.2.3 - Shopfloor KPIs / Analytics

The Shopfloor KPI/Analytics feature of the United Manufacturing Hub provides equipment-based KPIs, configurable dashboards, and detailed analytics for production transparency. Configure OEE calculation and track root causes of low OEE using drill-downs. Easily ingest, process, and analyze data in Grafana.

The Shopfloor KPI / Analytics feature of the United Manufacturing Hub provides a configurable and plug-and-play approach to create “Shopfloor Dashboards” for production transparency consisting of various KPIs and drill-downs.

Click on the images to enlarge them. More examples can be found in this YouTube video and in our community-repo on GitHub.

When should I use it?

If you want to create production dashboards that are highly configurable and can drill down into specific KPIs, the Shopfloor KPI / Analytics feature of the United Manufacturing Hub is an ideal choice. This feature is designed to help you quickly and easily create dashboards that provide a clear view of your shop floor performance.

What can I do with it?

The Shopfloor KPI / Analytics feature of the United Manufacturing Hub allows you to:

Query and visualize

In Grafana, you can:

  • Calculate the OEE (Overall Equipment Effectiveness) and view trends over time
    • Availability is calculated using the formula (plannedTime - stopTime) / plannedTime, where plannedTime is the duration of time for all machines states that do not belong in the Availability or Performance category, and stopTime is the duration of all machine states configured to be an availability stop.
    • Performance is calculated using the formula runningTime / (runningTime + stopTime), where runningTime is the duration of all machine states that consider the machine to be running, and stopTime is the duration of all machine states that are considered a performance loss. Note that this formula does not take into account losses caused by letting the machine run at a lower speed than possible. To approximate this, you can use the LowSpeedThresholdInPcsPerHour configuration option (see further below).
    • Quality is calculated using the formula good pieces / total pieces
  • Drill down into stop reasons (including histograms) to identify the root-causes for a potentially low OEE.
  • List all produced and planned orders including target vs actual produced pieces, total production time, stop reasons per order, and more using job and product tables.
  • See machine states, shifts, and orders on timelines to get a clear view of what happened during a specific time range.
  • View production speed and produced pieces over time.

Configure

In the database, you can configure:

  • Stop Reasons Configuration: Configure which stop reasons belong into which category for the OEE calculation and whether they should be included in the OEE calculation at all. For instance, some companies define changeovers as availability losses, some as performance losses. You can easily move them into the correct category.
  • Automatic Detection and Classification: Configure whether to automatically detect/classify certain types of machine states and stops:
    • AutomaticallyIdentifyChangeovers: If the machine state was an unspecified machine stop (UnknownStop), but an order was recently started, the time between the start of the order until the machine state turns to running, will be considered a Changeover Preparation State (10010). If this happens at the end of the order, it will be a Changeover Post-processing State (10020).
    • MicrostopDurationInSeconds: If an unspecified stop (UnknownStop) has a duration smaller than a configurable threshold (e.g., 120 seconds), it will be considered a Microstop State (50000) instead. Some companies put small unknown stops into a different category (performance) than larger unknown stops, which usually land up in the availability loss bucket.
    • IgnoreMicrostopUnderThisDurationInSeconds: In some cases, the machine can actually stop for a couple of seconds in routine intervals, which might be unwanted as it makes analysis difficult. One can set a threshold to ignore microstops that are smaller than a configurable threshold (usually like 1-2 seconds).
    • MinimumRunningTimeInSeconds: Same logic if the machine is running for a couple of seconds only. With this configurable threshold, small run-times can be ignored. These can happen, for example, during the changeover phase.
    • ThresholdForNoShiftsConsideredBreakInSeconds: If no shift was planned, an UnknownStop will always be classified as a NoShift state. Some companies move smaller NoShift’s into their category called “Break” and move them either into Availability or Performance.
    • LowSpeedThresholdInPcsPerHour: For a simplified performance calculation, a threshold can be set, and if the machine has a lower speed than this, it could be considered a LowSpeedState and could be categorized into the performance loss bucket.
  • Language Configuration: The language of the machine states can be configured using the languageCode configuration option (or overwritten in Grafana).

You can find the configuration options in the configurationTable

How can I use it?

Using it is very easy:

  1. Send messages according to the UMH datamodel to the Unified Namespace (similar to the Historian feature)
  2. Configure your OEE calculation by adjusting the configuration table
  3. Open Grafana, select your equipment and select the analysis you want to have. More information can be found in the umh-datasource-v2.

For more information about what exactly is behind the Analytics feature, check out our our architecture page and our datamodel

What are the limitations?

At the moment, the limitations are:

  • Speed losses in Performance are not calculated and can only be approximated using the LowSpeedThresholdInPcsPerHour configuration option
  • There is no way of tracking losses through reworked products. Either a product is scrapped or not.

Where to get more information?

2.2.4 - Alerting

Monitor and maintain your manufacturing processes with real-time Grafana alerts from the United Manufacturing Hub. Get notified of potential issues and reduce downtime by proactively addressing problems.

The United Manufacturing Hub utilizes a TimescaleDB database, which is based on PostgreSQL. Therefore, you can use the PostgreSQL plugin in Grafana to implement and configure alerts and notifications.

Why should I use it?

Alerts based on real-time data enable proactive problem detection. For example, you will receive a notification if the temperature of machine oil or an electrical component of a production line exceeds limitations. By utilizing such alerts, you can schedule maintenance, enhance efficiency, and reduce downtime in your factories.

What can I do with it?

Grafana alerts help you keep an eye on your production and manufacturing processes. By setting up alerts, you can quickly identify problems, ensuring smooth operations and high-quality products. An example of using alerts is the tracking of the temperature of an industrial oven. If the temperature goes too high or too low, you will get an alert, and the responsible team can take action before any damage occurs. Alerts can be configured in many different ways, for example, to set off an alarm if a maximum is reached once or if it exceeds a limit when averaged over a time period. It is also possible to include several values to create an alert, for example if a temperature surpasses a limit and/or the concentration of a component is too low. Notifications can be sent simultaneously across many services like Discord, Mail, Slack, Webhook, Telegram, or Microsoft Teams. It is also possible to forward the alert with SMS over a personal Webhook. A complete list can be found on the Grafana page about alerting.

How can I use it?

For a detailed tutorial on how to set up an alert, please visit our learn page with the detailed step-by-step tutorial. Here you can find an overview of the process.

  1. Install the PostgreSQL plugin in Grafana: Before you can formulate alerts, you need to install the PostgreSQL plugin, which is already integrated into Grafana.

  2. Alert Rule: When creating an alert, you first have to set the alert rule in Grafana. Here you set a name, specify which values are used for the rule, and when the rule is fired. Additionally, you can add labels for your rules, to link them to the correct contact points. You have to use SQL to select the desired values.

  3. Contact Point: In a contact point you create a collection of addresses and services that should be notified in case of an alert. This could be a Discord channel or Slack for example. When a linked alert is triggered, everyone within the contact point receives a message. The messages can be preconfigured and are specific to every service or contact.

  4. Notification Policies: In a notification policy, you establish the connection of a contact point with the desired alerts. This is done by adding the labels of the desired alerts and the contact point to the policy.

  5. Mute Timing: In case you do not want to receive messages during a recurring time period, you can add a mute timing to Grafana. If added to the notification policy, no notifications will be sent out by the contact point. This could be times without shifts, like weekends or during regular maintenance.

  6. Silence: You can also add silences for a specific time frame and labels, in case you only want to mute alerts once.

An alert is only sent out once after being triggered. For the next alert, it has to return to the normal state, so the data no longer violates the rule.

What are the limitations?

It can be complicated to select and manipulate the desired values to create the correct function for your application. Grafana cannot differentiate between data points of the same source. For example, you want to make a temperature threshold based on a single sensor. If your query selects the last three values and two of them are above the threshold, Grafana will fire two alerts which it cannot tell apart. This results in errors. You have to configure the rule to reduce the selected values to only one per source to avoid this. It can be complicated to create such a specific rule with this limitation, and it requires some testing.

Another thing to keep in mind is that the alerts can only work with data from the database. It also does not work with the machine status; these values only exist in a raw, unprocessed form in TimescaleDB and are not processed through an API like process values.

Where to get more information?

2.3 - Device & Container Infrastructure

This page describes the device and container infrastructure features of the United Manufacturing Hub.

2.3.1 - Layered Scaling

Efficiently scale your United Manufacturing Hub deployment across edge devices and servers using Layered Scaling.

Layered Scaling is an architectural approach in the United Manufacturing Hub that enables efficient scaling of your deployment across edge devices and servers. It is part of the Plant centric infrastructure , by dividing the processing workload across multiple layers or tiers, each with a specific set of responsibilities, Layered Scaling allows for better management of resources, improved performance, and easier deployment of software components. Layered Scaling follows the standard IoT infrastructure, by additionally connection a lot of IoT-devices typically via MQTT.

When should I use it?

Layered Scaling is ideal when:

  • You need to process and exchange data close to the source for latency reasons and independence from internet and network outages. For example, if you are taking pictures locally, analyzing them using machine learning, and then scrapping the product if the quality is poor. In this case, you don’t want the machine to be idle if something happens in the network. Also, it would not be acceptable for a message to arrive a few hundred milliseconds later, as the process is quicker than that.
  • High-frequency data might be useful to not send to the “higher” instance and store there. It can put unnecessary stress on those instances. You have an edge device that takes care of it. For example, you are taking and processing images (e.g., for quality reasons) or using an accelerometer and microphone for predictive maintenance reasons on the machine and do not want to send data streams with 20 kHz (20,000 times per second) to the next instance.
  • Organizational reasons. For the OT person, it might be better to configure the data contextualization using Node-RED directly at the production machine. They could experiment with it, configure it without endangering other machines, and see immediate results (e.g., when they move the position of a sensor). If the instance is “somewhere in IT,” they may feel they do not have control over it anymore and that it is not their system.

What can I do with it?

With Layered Scaling in the United Manufacturing Hub, you can:

  • Deploy minimized versions of the Helm Chart on edge devices, focusing on specific features required for that environment (e.g., without the Historian and Analytics features enabled, but with the IFM retrofitting feature using sensorconnect, with the barcodereader retrofit feature using barcodereader, or with the data connectivity via Node-RED feature enabled).
  • Seamlessly communicate between edge devices, on-premise servers, and cloud instances using the kafka-bridge microservice, allowing data to be buffered in between in case the internet or network connection drops.
  • Allow plant-overarching analysis / benchmarking, multi-plant kpis, connections to enterprise-IT, etc.. We typically recommend sending only data processed by our API factoryinsight.

How can I use it?

To implement Layered Scaling in the United Manufacturing Hub:

  1. Deploy a minimized version of the Helm Chart on edge devices, tailored to the specific features required for that environment. You can either install the whole version using flatcar and then disable functionalities you do not need, or use the Management Console. If the feature is not available in the Management Console, you could try asking nicely in the Discord and we will, can provide you with a token you can enter during the flatcar installation, so that your edge devices are pre-configured depending on your needs (incl. demilitarized zones, multiple networks, etc.)
  2. Deploy the full Helm Chart with all features enabled on a central instance, such as a server.
  3. Configure the Kafka Bridge microservice to transmit data from the edge devices to the central instance for further processing and analysis.

For MQTT connections, you can just connect external devices via MQTT, and it will land up in kafka directly. To connect on-premise servers with the cloud (plant-overarching architecture), you can use kafka-bridge or write service in benthos or Node-RED that regularly fetches data from factoryinsight and pushes it into your cloud instance.

What are the limitations?

  • Be aware that each device increases the complexity over the entire system. We recommend using the Management Console to manage them centrally.

Because Kafka is used to reliably transmit messages from the edge devices to the server, and it struggles with devices repeatedly going offline and online again, ethernet connections should be used. Also, the total amount of edge devices should not “escalate”. If you have a lot of edge devices (e.g., you want to connect each PLC), we recommend connecting them via MQTT to an instance of the UMH instead.

Where to get more information?

3 - Data Model

This page describes the data model of the UMH stack - from the message payloads up to database tables.

Raw Data

If you have events that you just want to send to the message broker / Unified Namespace without the need for it to be stored, simply send it to the raw topic. This data will not be processed by the UMH stack, but you can use it to build your own data processing pipeline.

ProcessValue Data

If you have data that does not fit in the other topics (such as your PLC tags or sensor data), you can use the processValue topic. It will be saved in the database in the processValue or processValueString and can be queried using factorysinsight or the umh-datasource Grafana plugin.

Production Data

In a production environment, you should first declare products using addProduct. This allows you to create an order using addOrder. Once you have created an order, send an state message to tell the database that the machine is working (or not working) on the order.

When the machine is ordered to produce a product, send a startOrder message. When the machine has finished producing the product, send an endOrder message.

Send count messages if the machine has produced a product, but it does not make sense to give the product its ID. Especially useful for bottling or any other use case with a large amount of products, where not each product is traced.

You can also add shifts using addShift.

All messages land up in different tables in the database and will be accessible from factorysinsight or the umh-datasource Grafana plugin.

Recommendation: Start with addShift and state and continue from there on

Modifying Data

If you have accidentally sent the wrong state or if you want to modify a value, you can use the modifyState message.

Unique Product Tracking

You can use uniqueProduct to tell the database that a new instance of a product has been created. If the produced product is scrapped, you can use scrapUniqueProduct to change its state to scrapped.

3.1 - Messages

For each message topic you will find a short description what the message is used for and which structure it has, as well as what structure the payload is excepted to have.

Introduction

The United Manufacturing Hub provides a specific structure for messages/topics, each with its own unique purpose. By adhering to this structure, the UMH will automatically calculate KPIs for you, while also making it easier to maintain consistency in your topic structure.

3.1.1 - activity

activity messages are sent when a new order is added.

This is part of our recommended workflow to create machine states. The data sent here will not be stored in the database automatically, as it will be required to be converted into a state. In the future, there will be a microservice, which converts these automatically.

Topic


ia/<customerID>/<location>/<AssetID>/activity



ia.<customerID>.<location>.<AssetID>.activity

Usage

A message is sent here each time the machine runs or stops.

Content

keydata typedescription
timestamp_msintunix timestamp of message creation
activitybooltrue if asset is currently active, false if asset is currently inactive

JSON

Examples

The asset was active during the timestamp of the message:

{
  "timestamp_ms":1588879689394,
  "activity": true,
}

Schema

Producers

  • Typically Node-RED

Consumers

  • Typically Node-RED

3.1.2 - addOrder

AddOrder messages are sent when a new order is added.

Topic


ia/<customerID>/<location>/<AssetID>/addOrder



ia.<customerID>.<location>.<AssetID>.addOrder

Usage

A message is sent here each time a new order is added.

Content

keydata typedescription
product_idstringcurrent product name
order_idstringcurrent order name
target_unitsint64amount of units to be produced
  1. The product needs to be added before adding the order. Otherwise, this message will be discarded
  2. One order is always specific to that asset and can, by definition, not be used across machines. For this case one would need to create one order and product for each asset (reason: one product might go through multiple machines, but might have different target durations or even target units, e.g. one big 100m batch get split up into multiple pieces)

JSON

Examples

One order was started for 100 units of product “test”:

{
  "product_id":"test",
  "order_id":"test_order",
  "target_units":100
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/addOrder.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "product_id",
        "order_id",
        "target_units"
    ],
    "properties": {
        "product_id": {
            "type": "string",
            "default": "",
            "title": "The product id to be produced",
            "examples": [
                "test",
                "Beierlinger 30x15"
            ]
        },
        "order_id": {
            "type": "string",
            "default": "",
            "title": "The order id of the order",
            "examples": [
                "test_order",
                "HA16/4889"
            ]
        },
        "target_units": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The amount of units to be produced",
            "examples": [
                1,
                100
            ]
        }
    },
    "examples": [{
      "product_id": "Beierlinger 30x15",
      "order_id": "HA16/4889",
      "target_units": 1
    },{
      "product_id":"test",
      "order_id":"test_order",
      "target_units":100
    }]
}

Producers

  • Typically Node-RED

Consumers

3.1.3 - addParentToChild

AddParentToChild messages are sent when child products are added to a parent product.

Topic


ia/<customerID>/<location>/<AssetID>/addParentToChild



ia.<customerID>.<location>.<AssetID>.addParentToChild

Usage

This message can be emitted to add a child product to a parent product. It can be sent multiple times, if a parent product is split up into multiple child’s or multiple parents are combined into one child. One example for this if multiple parts are assembled to a single product.

Content

keydata typedescription
timestamp_msint64unix timestamp you want to go back from
childAIDstringthe AID of the child product
parentAIDstringthe AID of the parent product

JSON

Examples

A parent is added to a child:

{
  "timestamp_ms":1589788888888,
  "childAID":"23948723489",
  "parentAID":"4329875"
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/scrapCount.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "timestamp_ms",
        "childAID",
        "parentAID"
    ],
    "properties": {
        "timestamp_ms": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The unix timestamp you want to go back from",
            "examples": [
              1589788888888
            ]
        },
        "childAID": {
            "type": "string",
            "default": "",
            "title": "The AID of the child product",
            "examples": [
              "23948723489"
            ]
        },
        "parentAID": {
            "type": "string",
            "default": "",
            "title": "The AID of the parent product",
            "examples": [
              "4329875"
            ]
        }
    },
    "examples": [
        {
            "timestamp_ms":1589788888888,
            "childAID":"23948723489",
            "parentAID":"4329875"
        },
        {
            "timestamp_ms":1589788888888,
            "childAID":"TestChild",
            "parentAID":"TestParent"
        }
    ]
}

Producers

  • Typically Node-RED

Consumers

3.1.4 - addProduct

AddProduct messages are sent when a new product is produced.

Topic


ia/<customerID>/<location>/<AssetID>/addProduct



ia.<customerID>.<location>.<AssetID>.addProduct

Usage

A message is sent each time a new product is produced.

Content

keydata typedescription
product_idstringcurrent product name
time_per_unit_in_secondsfloat64the time it takes to produce one unit of the product

See also notes regarding adding products and orders in /addOrder

JSON

Examples

A new product “Beilinger 30x15” with a cycle time of 200ms is added to the asset.

{
  "product_id": "Beilinger 30x15",
  "time_per_unit_in_seconds": "0.2"
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/scrapCount.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "product_id",
        "time_per_unit_in_seconds"
    ],
    "properties": {
        "product_id": {
          "type": "string",
          "default": "",
          "title": "The product id to be produced"
        },
        "time_per_unit_in_seconds": {
          "type": "number",
          "default": 0.0,
          "minimum": 0,
          "title": "The time it takes to produce one unit of the product"
        }
    },
    "examples": [
        {
            "product_id": "Beierlinger 30x15",
            "time_per_unit_in_seconds": "0.2"
        },
        {
            "product_id": "Test product",
            "time_per_unit_in_seconds": "10"
        }
    ]
}

Producers

  • Typically Node-RED

Consumers

3.1.5 - addShift

AddShift messages are sent to add a shift with start and end timestamp.

Topic


ia/<customerID>/<location>/<AssetID>/addShift



ia.<customerID>.<location>.<AssetID>.addShift

Usage

This message is send to indicate the start and end of a shift.

Content

keydata typedescription
timestamp_msint64unix timestamp of the shift start
timestamp_ms_endint64optional unix timestamp of the shift end

JSON

Examples

A shift with start and end:

{
  "timestamp_ms":1589788888888,
  "timestamp_ms_end":1589788888888
}

And shift without end:

{
  "timestamp_ms":1589788888888
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/scrapCount.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "timestamp_ms"
    ],
    "properties": {
        "timestamp_ms": {
            "type": "integer",
            "description": "The unix timestamp, of shift start"
        },
        "timestamp_ms_end": {
            "type": "integer",
            "description": "The *optional* unix timestamp, of shift end"
        }
    },
    "examples": [
        {
            "timestamp_ms":1589788888888,
            "timestamp_ms_end":1589788888888
        },
        {
            "timestamp_ms":1589788888888
        }
    ]
}

Producers

Consumers

3.1.6 - count

Count Messages are sent everytime an asset has counted a new item.

Topic


ia/<customerID>/<location>/<AssetID>/count



ia.<customerID>.<location>.<AssetID>.count

Usage

A count message is send everytime an asset has counted a new item.

Content

keydata typedescription
timestamp_msint64unix timestamp of message creation
countint64amount of items counted
scrapint64optional amount of defective items. In unset 0 is assumed

JSON

Examples

One item was counted and there was no scrap:

{
  "timestamp_ms":1589788888888,
  "count":1,
  "scrap":0
}

Ten items where counted and there was five scrap:

{
  "timestamp_ms":1589788888888,
  "count":10,
  "scrap":5
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/count.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "timestamp_ms",
        "count"
    ],
    "properties": {
        "timestamp_ms": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The unix timestamp of message creation",
            "examples": [
                1589788888888
            ]
        },
        "count": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The amount of items counted",
            "examples": [
                1
            ]
        },
        "scrap": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The optional amount of defective items",
            "examples": [
                0
            ]
        }
    },
    "examples": [{
      "timestamp_ms": 1589788888888,
      "count": 1,
      "scrap": 0
    },{
      "timestamp_ms": 1589788888888,
      "count": 1
    }]
}

Producers

  • Typically Node-RED

Consumers

3.1.7 - deleteShift

DeleteShift messages are sent to delete a shift that starts at the designated timestamp.

Topic


ia/<customerID>/<location>/<AssetID>/deleteShift



ia.<customerID>.<location>.<AssetID>.deleteShift

Usage

deleteShift is generated to delete a shift that started at the designated timestamp.

Content

keydata typedescription
timestamp_msint32unix timestamp of the shift start

JSON

Example

The shift that started at the designated timestamp is deleted from the database.

{
    "begin_time_stamp": 1588879689394
}

Producers

  • Typically Node-RED

Consumers

3.1.8 - detectedAnomaly

detectedAnomaly messages are sent when an asset has stopped and the reason is identified.

This is part of our recommended workflow to create machine states. The data sent here will not be stored in the database automatically, as it will be required to be converted into a state. In the future, there will be a microservice, which converts these automatically.

Topic


ia/<customerID>/<location>/<AssetID>/detectedAnomaly



ia.<customerID>.<location>.<AssetID>.detectedAnomaly

Usage

A message is sent here each time a stop reason has been identified automatically or by input from the machine operator.

Content

keydata typedescription
timestamp_msintUnix timestamp of message creation
detectedAnomalystringreason for the production stop of the asset

JSON

Examples

The anomaly of the asset has been identified as maintenance:

{
  "timestamp_ms":1588879689394,
  "detectedAnomaly":"maintenance",
}

Producers

  • Typically Node-RED

Consumers

  • Typically Node-RED

3.1.9 - endOrder

EndOrder messages are sent whenever a new product is produced.

Topic


ia/<customerID>/<location>/<AssetID>/endOrder



ia.<customerID>.<location>.<AssetID>.endOrder

Usage

A message is sent each time a new product is produced.

Content

keydata typedescription
timestamp_msint64unix timestamp of message creation
order_idint64current order name

See also notes regarding adding products and orders in /addOrder

JSON

Examples

The order “test_order” was finished at the shown timestamp.

{
  "order_id":"test_order",
  "timestamp_ms":1589788888888
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/endOrder.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "order_id",
        "timestamp_ms"
    ],
    "properties": {
        "timestamp_ms": {
          "type": "integer",
          "description": "The unix timestamp, of shift start"
        },
        "order_id": {
            "type": "string",
            "default": "",
            "title": "The order id of the order",
            "examples": [
                "test_order",
                "HA16/4889"
            ]
        }
    },
    "examples": [{
      "order_id": "HA16/4889",
      "timestamp_ms":1589788888888
    },{
      "product_id":"test",
      "timestamp_ms":1589788888888
    }]
}

Producers

  • Typically Node-RED

Consumers

3.1.10 - modifyProducedPieces

ModifyProducesPieces messages are sent whenever the count of produced and scrapped items need to be modified.

Topic


ia/<customerID>/<location>/<AssetID>/modifyProducedPieces



ia.<customerID>.<location>.<AssetID>.modifyProducedPieces

Usage

modifyProducedPieces is generated to change the count of produced items and scrapped items at the named timestamp.

Content

keydata typedescription
timestamp_msint64unix timestamp of the time point whose count is to be modified
countint32number of produced items
scrapint32number of scrapped items

JSON

Example

The count and scrap are overwritten to be to each at the timestamp.

{
    "timestamp_ms": 1588879689394,
    "count": 10,
    "scrap": 10
}

Producers

  • Typically Node-RED

Consumers

3.1.11 - modifyState

ModifyState messages are generated when a state of an asset during a certain timeframe needs to be modified.

Topic


ia/<customerID>/<location>/<AssetID>/modifyState



ia.<customerID>.<location>.<AssetID>.modifyState

Usage

modifyState is generated to modify the state from the starting timestamp to the end timestamp. You can find a list of all supported states here.

Content

keydata typedescription
timestamp_msint32unix timestamp of the starting point of the timeframe to be modified
timestamp_ms_endint32unix timestamp of the end point of the timeframe to be modified
new_stateint32new state code

JSON

Example

The state of the timeframe between the timestamp is modified to be 150000: OperatorBreakState

{
    "timestamp_ms": 1588879689394,
    "timestamp_ms_end": 1588891381023,
    "new_state": 150000
}

Producers

  • Typically Node-RED

Consumers

3.1.12 - processValue

ProcessValue messages are sent whenever a custom process value with unique name has been prepared. The value is numerical.

Topic


ia/<customerID>/<location>/<AssetID>/processValue 
or: ia/<customerID>/<location>/<AssetID>/processValue/<tagName>



ia.<customerID>.<location>.<AssetID>.processValue
or: ia.<customerID>.<location>.<AssetID>.processValue.<tagName>

If you have a lot of processValues, we’d recommend not using the /processValue as topic, but to append the tag name as well, e.g., /processValue/energyConsumption. This will structure it better for usage in MQTT Explorer or for data processing only certain processValues.

For automatic data storage in kafka-to-postgresql both will work fine as long as the payload is correct.

Please be aware that the values may only be int or float, other character are not valid, so make sure there is no quotation marks or anything sneaking in there. Also be cautious of using the JavaScript ToFixed() function, as it is converting a float into a string.

Usage

A message is sent each time a process value has been prepared. The key has a unique name.

Content

keydata typedescription
timestamp_msint64unix timestamp of message creation
<valuename>int64 or float64Represents a process value, e.g. temperature

Pre 0.10.0: As <valuename> is either of type ´int64´ or ´float64´, you cannot use booleans. Convert to integers as needed; e.g., true = “1”, false = “0”

Post 0.10.0: <valuename> will be converted, even if it is a boolean value. Check integer literals and floating-point literals for other valid values.

JSON

Example

At the shown timestamp the custom process value “energyConsumption” had a readout of 123456.

{
    "timestamp_ms": 1588879689394, 
    "energyConsumption": 123456
}

Producers

  • Typically Node-RED

Consumers

3.1.13 - processValueString

ProcessValueString messages are sent whenever a custom process value is prepared. The value is a string.

This message type is not functional as of 0.9.5!

Topic


ia/<customerID>/<location>/<AssetID>/processValueString



ia.<customerID>.<location>.<AssetID>.processValueString

Usage

A message is sent each time a process value has been prepared. The key has a unique name. This message is used when the datatype of the process value is a string instead of a number.

Content

keydata typedescription
timestamp_msint64unix timestamp of message creation
<valuename>stringRepresents a process value, e.g. temperature

JSON

Example

At the shown timestamp the custom process value “customer” had a readout of “miller”.

{
    "timestamp_ms": 1588879689394, 
    "customer": "miller"
}

Producers

  • Typically Node-RED

Consumers

3.1.14 - productTag

ProductTag messages are sent to contextualize processValue messages.

Topic


ia/<customerID>/<location>/<AssetID>/productTag



ia.<customerID>.<location>.<AssetID>.productTag

Usage

productTagString is usually generated by contextualizing a processValue.

Content

keydata typedescription
AIDstringAID of the product
namestringName of the product
valuefloat64key of the processValue
timestamp_msint64unix timestamp of message creation

JSON

Example

At the shown timestamp the product with the shown AID had 5 blemishes recorded.

{
    "AID": "43298756", 
    "name": "blemishes",
    "value": 5, 
    "timestamp_ms": 1588879689394
}

Producers

  • Typically Node-RED

Consumers

3.1.15 - productTagString

ProductTagString messages are sent to contextualize processValueString messages.

Topic


ia/<customerID>/<location>/<AssetID>/productTagString



ia.<customerID>.<location>.<AssetID>.productTagString

Usage

ProductTagString is usually generated by contextualizing a processValueString.

Content

keydata typedescription
AIDstringAID of the product
namestringKey of the processValue
valuestringvalue of the processValue
timestamp_msint64unix timestamp of message creation

JSON

Example

At the shown timestamp the product with the shown AID had the processValue of “test_value”.

{
    "AID": "43298756", 
    "name": "shirt_size",
    "value": "XL", 
    "timestamp_ms": 1588879689394
}

Producers

Consumers

3.1.16 - recommendation

Recommendation messages are sent whenever rapid actions would quickly improve efficiency on the shop floor.

Topic


ia/<customerID>/<location>/<AssetID>/recommendation



ia.<customerID>.<location>.<AssetID>.recommendation

Usage

recommendation are action recommendations, which require concrete and rapid action in order to quickly eliminate efficiency losses on the store floor.

Content

keydata typedescription
uidstringUniqueID of the product
timestamp_msint64unix timestamp of message creation
customerstringthe customer ID in the data structure
locationstringthe location in the data structure
assetstringthe asset ID in the data structure
recommendationTypeint32Name of the product
enabledbool-
recommendationValuesmapMap of values based on which this recommendation is created
diagnoseTextDEstringDiagnosis of the recommendation in german
diagnoseTextENstringDiagnosis of the recommendation in english
recommendationTextDEstringRecommendation in german
recommendationTextENstringRecommendation in english

JSON

Example

A recommendation for the demonstrator at the shown location has not been running for a while, so a recommendation is sent to either start the machine or specify a reason why it is not running.

{
    "UID": "43298756", 
    "timestamp_ms": 15888796894,
    "customer": "united-manufacturing-hub",
    "location": "dccaachen", 
    "asset": "DCCAachen-Demonstrator",
    "recommendationType": "1", 
    "enabled": true,
    "recommendationValues": { "Treshold": 30, "StoppedForTime": 612685 }, 
    "diagnoseTextDE": "Maschine DCCAachen-Demonstrator steht seit 612685 Sekunden still (Status: 8, Schwellwert: 30)" ,
    "diagnoseTextEN": "Machine DCCAachen-Demonstrator is not running since 612685 seconds (status: 8, threshold: 30)", 
    "recommendationTextDE":"Maschine DCCAachen-Demonstrator einschalten oder Stoppgrund auswählen.",
    "recommendationTextEN": "Start machine DCCAachen-Demonstrator or specify stop reason.", 
}

Producers

  • Typically Node-RED

Consumers

3.1.17 - scrapCount

ScrapCount messages are sent whenever a product is to be marked as scrap.

Topic


ia/<customerID>/<location>/<AssetID>/scrapCount



ia.<customerID>.<location>.<AssetID>.scrapCount

Usage

Here a message is sent every time products should be marked as scrap. It works as follows: A message with scrap and timestamp_ms is sent. It starts with the count that is directly before timestamp_ms. It is now iterated step by step back in time and step by step the existing counts are set to scrap until a total of scrap products have been scraped.

Content

  • timestamp_ms is the unix timestamp, you want to go back from
  • scrap number of item to be considered as scrap.
  1. You can specify maximum of 24h to be scrapped to avoid accidents
  2. (NOT IMPLEMENTED YET) If counts does not equal scrap, e.g. the count is 5 but only 2 more need to be scrapped, it will scrap exactly 2. Currently, it would ignore these 2. see also #125
  3. (NOT IMPLEMENTED YET) If no counts are available for this asset, but uniqueProducts are available, they can also be marked as scrap.

JSON

Examples

Ten items where scrapped:

{
  "timestamp_ms":1589788888888,
  "scrap":10
}

Schema

{
    "$schema": "http://json-schema.org/draft/2019-09/schema",
    "$id": "https://learn.umh.app/content/docs/architecture/datamodel/messages/scrapCount.json",
    "type": "object",
    "default": {},
    "title": "Root Schema",
    "required": [
        "timestamp_ms",
        "scrap"
    ],
    "properties": {
        "timestamp_ms": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "The unix timestamp you want to go back from",
            "examples": [
              1589788888888
            ]
        },
        "scrap": {
            "type": "integer",
            "default": 0,
            "minimum": 0,
            "title": "Number of items to be considered as scrap",
            "examples": [
                10
            ]
        }
    },
    "examples": [
        {
            "timestamp_ms": 1589788888888,
            "scrap": 10
        },
        {
            "timestamp_ms": 1589788888888,
            "scrap": 5
        }
    ]
}

Producers

  • Typically Node-RED

Consumers

3.1.18 - scrapUniqueProduct

ScrapUniqueProduct messages are sent whenever a unique product should be scrapped.

Topic


ia/<customerID>/<location>/<AssetID>/scrapUniqueProduct



ia.<customerID>.<location>.<AssetID>.scrapUniqueProduct

Usage

A message is sent here everytime a unique product is scrapped.

Content

keydata typedescription
UIDstringunique ID of the current product

JSON

Example

The product with the unique ID 22 is scrapped.

{
    "UID": 22, 
}

Producers

  • Typically Node-RED

Consumers

3.1.19 - startOrder

StartOrder messages are sent whenever a new order is started.

Topic


ia/<customerID>/<location>/<AssetID>/startOrder



ia.<customerID>.<location>.<AssetID>.startOrder

Usage

A message is sent here everytime a new order is started.

Content

keydata typedescription
order_idstringname of the order
timestamp_msint64unix timestamp of message creation
  1. See also notes regarding adding products and orders in /addOrder
  2. When startOrder is executed multiple times for an order, the last used timestamp is used.

JSON

Example

The order “test_order” is started at the shown timestamp.

{
  "order_id":"test_order",
  "timestamp_ms":1589788888888
}

Producers

  • Typically Node-RED

Consumers

3.1.20 - state

State messages are sent every time an asset changes status.

Topic


ia/<customerID>/<location>/<AssetID>/state



ia.<customerID>.<location>.<AssetID>.state

Usage

A message is sent here each time the asset changes status. Subsequent changes are not possible. Different statuses can also be process steps, such as “setup”, “post-processing”, etc. You can find a list of all supported states here.

Content

keydata typedescription
stateuint32value of the state according to the link above
timestamp_msuint64unix timestamp of message creation

JSON

Example

The asset has a state of 10000, which means it is actively producing.

{
  "timestamp_ms":1589788888888,
  "state":10000
}

Producers

  • Typically Node-RED

Consumers

3.1.21 - uniqueProduct

UniqueProduct messages are sent whenever a unique product was produced or modified.

Topic


ia/<customerID>/<location>/<AssetID>/uniqueProduct



ia.<customerID>.<location>.<AssetID>.uniqueProduct

Usage

A message is sent here each time a product has been produced or modified. A modification can take place, for example, due to a downstream quality control.

There are two cases of when to send a message under the uniqueProduct topic:

  • The exact product doesn’t already have a UID (-> This is the case, if it has not been produced at an asset incorporated in the digital shadow). Specify a space holder asset = “storage” in the MQTT message for the uniqueProduct topic.
  • The product was produced at the current asset (it is now different from before, e.g. after machining or after something was screwed in). The newly produced product is always the “child” of the process. Products it was made out of are called the “parents”.

Content

keydata typedescription
begin_timestamp_msint64unix timestamp of start time
end_timestamp_msint64unix timestamp of completion time
product_idstringproduct ID of the currently produced product
isScrapbooloptional information whether the current product is of poor quality and will be sorted out. Is considered false if not specified.
uniqueProductAlternativeIDstringalternative ID of the product

JSON

Example

The processing of product “Beilinger 30x15” with the AID 216381 started and ended at the designated timestamps. It is of low quality and due to be scrapped.

{
  "begin_timestamp_ms":1589788888888,
  "end_timestamp_ms":1589788893729,
  "product_id":"Beilinger 30x15",
  "isScrap":true,
  "uniqueProductAlternativeID":"216381"
}

Producers

  • Typically Node-RED

Consumers

3.2 - Database

The database stores the messages in different tables.

Introduction

We are using the database TimescaleDB, which is based on PostgreSQL and supports standard relational SQL database work, while also supporting time-series databases. This allows for usage of regular SQL queries, while also allowing to process and store time-series data. Postgresql has proven itself reliable over the last 25 years, so we are happy to use it.

If you want to learn more about database paradigms, please refer to the knowledge article about that topic. It also includes a concise video summarizing what you need to know about different paradigms.

Our database model is designed to represent a physical manufacturing process. It keeps track of the following data:

  • The state of the machine
  • The products that are produced
  • The orders for the products
  • The workers’ shifts
  • Arbitrary process values (sensor data)
  • The producible products
  • Recommendations for the production

Please note that our database does not use a retention policy. This means that your database can grow quite fast if you save a lot of process values. Take a look at our guide on enabling data compression and retention in TimescaleDB to customize the database to your needs.

A good method to check your db size would be to use the following commands inside postgres shell:

SELECT pg_size_pretty(pg_database_size('factoryinsight'));

3.2.1 - assetTable

assetTable is contains all assets and their location.

Usage

Primary table for our data structure, it contains all the assets and their location.

Structure

keydata typedescriptionexample
idintAuto incrementing id of the asset0
assetIDtextAsset namePrinter-03
locationtextPhysical location of the assetDCCAachen
customertextCustomer name, in most cases “factoryinsight”factoryinsight

Relations

assetTable
assetTable

DDL

 CREATE TABLE IF NOT EXISTS assetTable
 (
     id         SERIAL  PRIMARY KEY,
     assetID    TEXT    NOT NULL,
     location   TEXT    NOT NULL,
     customer   TEXT    NOT NULL,
     unique (assetID, location, customer)
 );

3.2.2 - configurationTable

configurationTable stores the configuration of the UMH system.

Usage

This table stores the configuration of the system

Structure

keydata typedescriptionexample
customertextCustomer namefactoryinsight
MicrostopDurationInSecondsintegerStop counts as microstop if smaller than this value120
IgnoreMicrostopUnderThisDurationInSecondsintegerIgnore stops under this value-1
MinimumRunningTimeInSecondsintegerMinimum runtime of the asset before tracking micro-stops0
ThresholdForNoShiftsConsideredBreakInSecondsintegerIf no shift is shorter than this value, it is a break2100
LowSpeedThresholdInPcsPerHourintegerThreshold once machine should go into low speed state-1
AutomaticallyIdentifyChangeoversbooleanAutomatically identify changeovers in productiontrue
LanguageCodeinteger0 is german, 1 is english1
AvailabilityLossStatesinteger[]States to count as availability loss{40000, 180000, 190000, 200000, 210000, 220000}
PerformanceLossStatesinteger[]States to count as performance loss{20000, 50000, 60000, 70000, 80000, 90000, 100000, 110000, 120000, 130000, 140000, 150000}

Relations

configurationTable
configurationTable

DDL

CREATE TABLE IF NOT EXISTS configurationTable
(
    customer TEXT PRIMARY KEY,
    MicrostopDurationInSeconds INTEGER DEFAULT 60*2,
    IgnoreMicrostopUnderThisDurationInSeconds INTEGER DEFAULT -1, --do not apply
    MinimumRunningTimeInSeconds INTEGER DEFAULT 0, --do not apply
    ThresholdForNoShiftsConsideredBreakInSeconds INTEGER DEFAULT 60*35,
    LowSpeedThresholdInPcsPerHour INTEGER DEFAULT -1, --do not apply
    AutomaticallyIdentifyChangeovers BOOLEAN DEFAULT true,
    LanguageCode INTEGER DEFAULT 1, -- english
    AvailabilityLossStates INTEGER[] DEFAULT '{40000, 180000, 190000, 200000, 210000, 220000}',
    PerformanceLossStates INTEGER[] DEFAULT '{20000, 50000, 60000, 70000, 80000, 90000, 100000, 110000, 120000, 130000, 140000, 150000}'
);

3.2.3 - countTable

countTable contains all reported counts of all assets.

Usage

This table contains all reported counts of the assets.

Structure

keydata typedescriptionexample
timestamptimestamptzEntry timestamp0
asset_idserialAsset id (see assetTable)1
countintegerA count greater 01

Relations

countTable
countTable

DDL

CREATE TABLE IF NOT EXISTS countTable
(
    timestamp                TIMESTAMPTZ                         NOT NULL,
    asset_id            SERIAL REFERENCES assetTable (id),
    count INTEGER CHECK (count > 0),
    UNIQUE(timestamp, asset_id)
);
-- creating hypertable
SELECT create_hypertable('countTable', 'timestamp');

-- creating an index to increase performance
CREATE INDEX ON countTable (asset_id, timestamp DESC);

3.2.4 - orderTable

orderTable contains orders for production.

Usage

This table stores orders for product production

Structure

keydata typedescriptionexample
order_idserialAuto incrementing id0
order_nametextName of the orderScarjit-500-DaVinci-1-24062022
product_idserialProduct id to produce1
begin_timestamptimestamptzBegin timestamp of the order0
end_timestamptimestamptzEnd timestamp of the order10000
target_unitsintegerHow many product to produce500
asset_idserialWhich asset to produce on (see assetTable)1

Relations

orderTable
orderTable

DDL

CREATE TABLE IF NOT EXISTS orderTable
(
    order_id        SERIAL          PRIMARY KEY,
    order_name      TEXT            NOT NULL,
    product_id      SERIAL          REFERENCES productTable (product_id),
    begin_timestamp TIMESTAMPTZ,
    end_timestamp   TIMESTAMPTZ,
    target_units    INTEGER,
    asset_id        SERIAL          REFERENCES assetTable (id),
    unique (asset_id, order_name),
    CHECK (begin_timestamp < end_timestamp),
    CHECK (target_units > 0),
    EXCLUDE USING gist (asset_id WITH =, tstzrange(begin_timestamp, end_timestamp) WITH &&) WHERE (begin_timestamp IS NOT NULL AND end_timestamp IS NOT NULL)
);

3.2.5 - processValueStringTable

processValueStringTable contains process values.

Usage

This table stores process values, for example toner level of a printer, flow rate of a pump, etc. This table, has a closely related table for storing number values, processValueTable.

Structure

keydata typedescriptionexample
timestamptimestamptzEntry timestamp0
asset_idserialAsset id (see assetTable)1
valueNametextName of the process valuetoner-level
valuestringValue of the process value100

Relations

processValueTable
processValueTable

DDL

CREATE TABLE IF NOT EXISTS processValueStringTable
(
    timestamp               TIMESTAMPTZ                         NOT NULL,
    asset_id                SERIAL                              REFERENCES assetTable (id),
    valueName               TEXT                                NOT NULL,
    value                   TEST                                NULL,
    UNIQUE(timestamp, asset_id, valueName)
);
-- creating hypertable
SELECT create_hypertable('processValueStringTable', 'timestamp');

-- creating an index to increase performance
CREATE INDEX ON processValueStringTable (asset_id, timestamp DESC);

-- creating an index to increase performance
CREATE INDEX ON processValueStringTable (valuename);

3.2.6 - processValueTable

processValueTable contains process values.

Usage

This table stores process values, for example toner level of a printer, flow rate of a pump, etc. This table, has a closely related table for storing string values, processValueStringTable.

Structure

keydata typedescriptionexample
timestamptimestamptzEntry timestamp0
asset_idserialAsset id (see assetTable)1
valueNametextName of the process valuetoner-level
valuedoubleValue of the process value100

Relations

processValueTable
processValueTable

DDL

CREATE TABLE IF NOT EXISTS processValueTable
(
    timestamp               TIMESTAMPTZ                         NOT NULL,
    asset_id                SERIAL                              REFERENCES assetTable (id),
    valueName               TEXT                                NOT NULL,
    value                   DOUBLE PRECISION                    NULL,
    UNIQUE(timestamp, asset_id, valueName)
);
-- creating hypertable
SELECT create_hypertable('processValueTable', 'timestamp');

-- creating an index to increase performance
CREATE INDEX ON processValueTable (asset_id, timestamp DESC);

-- creating an index to increase performance
CREATE INDEX ON processValueTable (valuename);

3.2.7 - productTable

productTable contains products in production.

Usage

This table products to be produced at assets

Structure

keydata typedescriptionexample
product_idserialAuto incrementing id0
product_nametextName of the productPainting-DaVinci-1
asset_idserialAsset producing this product (see assetTable)1
time_per_unit_in_secondsrealTime in seconds to produce this product600

Relations

productTable
productTable

DDL

CREATE TABLE IF NOT EXISTS productTable
(
    product_id                  SERIAL PRIMARY KEY,
    product_name                TEXT NOT NULL,
    asset_id                    SERIAL REFERENCES assetTable (id),
    time_per_unit_in_seconds    REAL NOT NULL,
    UNIQUE(product_name, asset_id),
    CHECK (time_per_unit_in_seconds > 0)
);

3.2.8 - recommendationTable

recommendationTable contains given recommendation for the shop floor assets.

Usage

This table stores recommendations

Structure

keydata typedescriptionexample
uidtextId of the recommendationrefill_toner
timestamptimestamptzTimestamp of recommendation insertion1
recommendationTypeintegerUsed to subscribe people to specific types only3
enabledboolRecommendation can be outputtedtrue
recommendationValuestextValues to change to resolve recommendation{ “toner-level”: 100 }
diagnoseTextDEtextDiagnose text in german“Der Toner ist leer”
diagnoseTextENtextDiagnose text in english“The toner is empty”
recommendationTextDEtextRecommendation text in german“Bitte den Toner auffüllen”
recommendationTextENtextRecommendation text in english“Please refill the toner”

Relations

recommendationTable
recommendationTable

DDL

CREATE TABLE IF NOT EXISTS recommendationTable
(
    uid                     TEXT                                PRIMARY KEY,
    timestamp               TIMESTAMPTZ                         NOT NULL,
    recommendationType      INTEGER                             NOT NULL,
    enabled                 BOOLEAN                             NOT NULL,
    recommendationValues    TEXT,
    diagnoseTextDE          TEXT,
    diagnoseTextEN          TEXT,
    recommendationTextDE    TEXT,
    recommendationTextEN    TEXT
);

3.2.9 - shiftTable

shiftTable contains shifts with asset, start and finish timestamp

Usage

This table stores shifts

Structure

keydata typedescriptionexample
idserialAuto incrementing id0
typeintegerShift type (1 for shift, 0 for no shift)1
begin_timestamptimestamptzBegin of the shift3
end_timestamptimestamptzEnd of the shift10
asset_idtextAsset ID the shift is performed on (see assetTable)1

Relations

shiftTable
shiftTable

DDL

-- Using btree_gist to avoid overlapping shifts
-- Source: https://gist.github.com/fphilipe/0a2a3d50a9f3834683bf
CREATE EXTENSION btree_gist;
CREATE TABLE IF NOT EXISTS shiftTable
(
    id              SERIAL      PRIMARY KEY,
    type            INTEGER,
    begin_timestamp TIMESTAMPTZ NOT NULL,
    end_timestamp   TIMESTAMPTZ,
    asset_id        SERIAL      REFERENCES assetTable (id),
    unique (begin_timestamp, asset_id),
    CHECK (begin_timestamp < end_timestamp),
    EXCLUDE USING gist (asset_id WITH =, tstzrange(begin_timestamp, end_timestamp) WITH &&)
);

3.2.10 - stateTable

stateTable contains the states of all assets.

Usage

This table contains all state changes of the assets.

Structure

keydata typedescriptionexample
timestamptimestamptzEntry timestamp0
asset_idserialAsset ID (see assetTable)1
stateintegerState ID (see states)40000

Relations

stateTable
stateTable

DDL

CREATE TABLE IF NOT EXISTS stateTable
(
    timestamp   TIMESTAMPTZ NOT NULL,
    asset_id    SERIAL      REFERENCES assetTable (id),
    state       INTEGER     CHECK (state >= 0),
    UNIQUE(timestamp, asset_id)
);
-- creating hypertable
SELECT create_hypertable('stateTable', 'timestamp');

-- creating an index to increase performance
CREATE INDEX ON stateTable (asset_id, timestamp DESC);

3.2.11 - uniqueProductTable

uniqueProductTable contains unique products and their IDs.

Usage

This table stores unique products.

Structure

keydata typedescriptionexample
uidtextID of a unique product0
asset_idserialAsset id (see assetTable)1
begin_timestamp_mstimestamptzTime when product was inputted in asset0
end_timestamp_mstimestamptzTime when product was output of asset100
product_idtextID of the product (see productTable)1
is_scrapbooleanTrue if product is scraptrue
quality_classtextQuality class of the productA
station_idtextID of the station where the product was processedSoldering Iron-1

Relations

uniqueProductTable
uniqueProductTable

DDL

CREATE TABLE IF NOT EXISTS uniqueProductTable
(
    uid                 TEXT        NOT NULL,
    asset_id            SERIAL      REFERENCES assetTable (id),
    begin_timestamp_ms  TIMESTAMPTZ NOT NULL,
    end_timestamp_ms    TIMESTAMPTZ NOT NULL,
    product_id          TEXT        NOT NULL,
    is_scrap            BOOLEAN     NOT NULL,
    quality_class       TEXT        NOT NULL,
    station_id          TEXT        NOT NULL,
    UNIQUE(uid, asset_id, station_id),
    CHECK (begin_timestamp_ms < end_timestamp_ms)
);

-- creating an index to increase performance
CREATE INDEX ON uniqueProductTable (asset_id, uid, station_id);

3.3 - States

States are the core of the database model. They represent the state of the machine at a given point in time.

States Documentation Index

Introduction

This documentation outlines the various states used in the United Manufacturing Hub software stack to calculate OEE/KPI and other production metrics.

State Categories

Glossary

  • OEE: Overall Equipment Effectiveness
  • KPI: Key Performance Indicator

Conclusion

This documentation provides a comprehensive overview of the states used in the United Manufacturing Hub software stack and their respective categories. For more information on each state category and its individual states, please refer to the corresponding subpages.

3.3.1 - Active (10000-29999)

These states represent that the asset is actively producing

10000: ProducingAtFullSpeedState

This asset is running at full speed.

Examples for ProducingAtFullSpeedState

  • WS_Cur_State: Operating
  • PackML/Tobacco: Execute

20000: ProducingAtLowerThanFullSpeedState

Asset is producing, but not at full speed.

Examples for ProducingAtLowerThanFullSpeedState

  • WS_Cur_Prog: StartUp
  • WS_Cur_Prog: RunDown
  • WS_Cur_State: Stopping
  • PackML/Tobacco : Stopping
  • WS_Cur_State: Aborting
  • PackML/Tobacco: Aborting
  • WS_Cur_State: Holding
  • Ws_Cur_State: Unholding
  • PackML:Tobacco: Unholding
  • WS_Cur_State Suspending
  • PackML/Tobacco: Suspending
  • WS_Cur_State: Unsuspending
  • PackML/Tobacco: Unsuspending
  • PackML/Tobacco: Completing
  • WS_Cur_Prog: Production
  • EUROMAP: MANUAL_RUN
  • EUROMAP: CONTROLLED_RUN

Currently not included:

  • WS_Prog_Step: all

3.3.2 - Unknown (30000-59999)

These states represent that the asset is in an unspecified state

30000: UnknownState

Data for that particular asset is not available (e.g. connection to the PLC is disrupted)

Examples for UnknownState

  • WS_Cur_Prog: Undefined
  • EUROMAP: Offline

40000 UnspecifiedStopState

The asset is not producing, but the reason is unknown at the time.

Examples for UnspecifiedStopState

  • WS_Cur_State: Clearing
  • PackML/Tobacco: Clearing
  • WS_Cur_State: Emergency Stop
  • WS_Cur_State: Resetting
  • PackML/Tobacco: Clearing
  • WS_Cur_State: Held
  • EUROMAP: Idle
  • Tobacco: Other
  • WS_Cur_State: Stopped
  • PackML/Tobacco: Stopped
  • WS_Cur_State: Starting
  • PackML/Tobacco: Starting
  • WS_Cur_State: Prepared
  • WS_Cur_State: Idle
  • PackML/Tobacco: Idle
  • PackML/Tobacco: Complete
  • EUROMAP: READY_TO_RUN

50000: MicrostopState

The asset is not producing for a short period (typically around five minutes), but the reason is unknown at the time.

3.3.3 - Material (60000-99999)

These states represent that the asset has issues regarding materials.

60000 InletJamState

This machine does not perform its intended function due to a lack of material flow in the infeed of the machine, detected by the sensor system of the control system (machine stop). In the case of machines that have several inlets, the condition o lack in the inlet refers to the main flow , i.e. to the material (crate, bottle) that is fed in the direction of the filling machine (Central machine). The defect in the infeed is an extraneous defect, but because of its importance for visualization and technical reporting, it is recorded separately.

Examples for InletJamState

  • WS_Cur_State: Lack

70000: OutletJamState

The machine does not perform its intended function as a result of a jam in the good flow discharge of the machine, detected by the sensor system of the control system (machine stop). In the case of machines that have several discharges, the jam in the discharge condition refers to the main flow, i.e. to the good (crate, bottle) that is fed in the direction of the filling machine (central machine) or is fed away from the filling machine. The jam in the outfeed is an external fault 1v, but it is recorded separately, because of its importance for visualization and technical reporting.

Examples for OutletJamState

  • WS_Cur_State: Tailback

80000: CongestionBypassState

The machine does not perform its intended function due to a shortage in the bypass supply or a jam in the bypass discharge of the machine, detected by the sensor system of the control system (machine stop). This condition can only occur in machines with two outlets or inlets and in which the bypass is in turn the inlet or outlet of an upstream or downstream machine of the filling line (packaging and palleting machines). The jam/shortage in the auxiliary flow is an external fault, but it is recoded separately due to its importance for visualization and technical reporting.

Examples for the CongestionBypassState

  • WS_Cur_State: Lack/Tailback Branch Line

90000: MaterialIssueOtherState

The asset has a material issue, but it is not further specified.

Examples for MaterialIssueOtherState

  • WS_Mat_Ready (Information of which material is lacking)
  • PackML/Tobacco: Suspended

3.3.4 - Process(100000-139999)

These states represent that the asset is in a stop, which belongs to the process and cannot be avoided.

100000: ChangeoverState

The asset is in a changeover process between products.

Examples for ChangeoverState

  • WS_Cur_Prog: Program-Changeover
  • Tobacco: CHANGE OVER

110000: CleaningState

The asset is currently in a cleaning process.

Examples for CleaningState

  • WS_Cur_Prog: Program-Cleaning
  • Tobacco: CLEAN
120000: EmptyingState

The asset is currently emptied, e.g. to prevent mold for food products over the long breaks, e.g. the weekend.

Examples for EmptyingState
  • Tobacco: EMPTY OUT

130000: SettingUpState

This machine is currently preparing itself for production, e.g. heating up.

Examples for SettingUpState
  • EUROMAP: PREPARING

3.3.5 - Operator (140000-159999)

These states represent that the asset is stopped because of operator related issues.

140000: OperatorNotAtMachineState

The operator is not at the machine.

150000: OperatorBreakState

The operator is taking a break.

This is different from a planned shift as it could contribute to performance losses.

Examples for OperatorBreakState

  • WS_Cur_Prog: Program-Break

3.3.6 - Planning (160000-179999)

These states represent that the asset is stopped as it is planned to stopped (planned idle time).

160000: NoShiftState

There is no shift planned at that asset.

170000: NO OrderState

There is no order planned at that asset.

3.3.7 - Technical (180000-229999)

These states represent that the asset has a technical issue.

180000: EquipmentFailureState

The asset itself is defect, e.g. a broken engine.

Examples for EquipmentFailureState

  • WS_Cur_State: Equipment Failure

190000: ExternalFailureState

There is an external failure, e.g. missing compressed air.

Examples for ExternalFailureState

  • WS_Cur_State: External Failure

200000: ExternalInterferenceState

There is an external interference, e.g. the crane to move the material is currently unavailable.

210000: PreventiveMaintenanceStop

A planned maintenance action.

Examples for PreventiveMaintenanceStop

  • WS_Cur_Prog: Program-Maintenance
  • PackML: Maintenance
  • EUROMAP: MAINTENANCE
  • Tobacco: MAINTENANCE

220000: TechnicalOtherStop

The asset has a technical issue, but it is not specified further.

Examples for TechnicalOtherStop

  • WS_Not_Of_Fail_Code
  • PackML: Held
  • EUROMAP: MALFUNCTION
  • Tobacco: MANUAL
  • Tobacco: SET UP
  • Tobacco: REMOTE SERVICE

4 - Architecture

A comprehensive overview of the United Manufacturing Hub architecture, detailing its deployment, management, and data processing capabilities.

The United Manufacturing Hub is a comprehensive Helm Chart for Kubernetes, integrating a variety of open source software, including notable third-party applications such as Node-RED and Grafana. Designed for versatility, UMH is deployable across a wide spectrum of environments, from edge devices to virtual machines, and even managed Kubernetes services, catering to diverse industrial needs.

The following diagram depicts the interaction dynamics between UMH’s components and user types, offering a visual guide to its architecture and operational mechanisms.

graph LR subgraph group1 [United Manufacturing Hub] style group1 fill:#ffffff,stroke:#47a0b5,color:#47a0b5,stroke-dasharray:5 16["`**Management Console** Configures, manages, and monitors Data and Device & Container Infrastructures in the UMH Integrated Platform`"] style 16 fill:#aaaaaa,stroke:#47a0b5,color:#000000 27["`**Device & Container Infrastructure** Oversees automated, streamlined installation of key software and operating systems`"] style 27 fill:#aaaaaa,stroke:#47a0b5,color:#000000 50["`**Data Infrastructure** Integrates every ISA-95 standard layer with the Unified Namespace, adding data sources beyond typical automation pyramid bounds`"] style 50 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 1["`fa:fa-user **IT/OT Professional** Manages and monitors the United Manufacturing Hub`"] style 1 fill:#dddddd,stroke:#9a9a9a,color:#000000 2["`fa:fa-user **OT Professional / Shopfloor** Monitors and manages the shopfloor, including safety, automation and maintenance`"] style 2 fill:#dddddd,stroke:#9a9a9a,color:#000000 3["`fa:fa-user **Business Analyst** Gathers and analyzes company data to identify needs and recommend solutions`"] style 3 fill:#dddddd,stroke:#9a9a9a,color:#000000 4["`**Data Warehouse/Data Lake** Stores data for analysis, on-premise or in the cloud`"] style 4 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 5["`**Automation Pyramid** Represents the layered structure of systems in manufacturing operations based on the ISA-95 model`"] style 5 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 1-. Interacts with the entire infrastructure .->16 16-. Manages & monitors .->27 16-. Manages & monitors .->50 2-. Access real-time dashboards from .->50 2-. Works with .->5 3-. Gets and analyzes data from .->4 50-. Is installed on .->27 50-. Provides data to .->4 50-. Provides to and extracts data from .->5

Management Console

The Management Console of the United Manufacturing Hub is a robust web application designed to configure, manage, and monitor the various aspects of Data and Device & Container Infrastructures within UMH. Acting as the central command center, it provides a comprehensive overview and control over the system’s functionalities, ensuring efficient operation and maintenance. The console simplifies complex processes, making it accessible for users to oversee the vast array of services and operations integral to UMH.

Device & Container Infrastructure

The Device & Container Infrastructure lays the foundation of the United Manufacturing Hub’s architecture, streamlining the deployment and setup of essential software and operating systems across devices. This infrastructure is pivotal in automating the installation process, ensuring that the essential software components and operating systems are efficiently and reliably established. It provides the groundwork upon which the Data Infrastructure is built, embodying a robust and scalable base for the entire architecture.

Data Infrastructure

The Data Infrastructure is the heart of the United Manufacturing Hub, orchestrating the interconnection of data sources, storage, monitoring, and analysis solutions. It comprises three key components:

  • Data Connectivity: Facilitates the integration of diverse data sources into UMH, enabling uninterrupted data exchange.
  • Unified Namespace (UNS): Centralizes and standardizes data within UMH into a cohesive model, by linking each layer of the ISA-95 automation pyramid to the UNS and assimilating non-traditional data sources.
  • Historian: Stores data in TimescaleDB, a PostgreSQL-based time-series database, allowing real-time and historical data analysis through Grafana or other tools.

The UMH Data Infrastructure leverages Industrial IoT to expand the ISA95 Automation Pyramid, enabling high-speed data processing using systems like Kafka. It enhances system availability through Kubernetes and simplifies maintenance with Docker and Prometheus. Additionally, it facilitates the use of AI, predictive maintenance, and digital twin technologies

Expandability

The United Manufacturing Hub is architecturally designed for high expandability, enabling integration of custom microservices or Docker containers. This adaptability allows for users to establish connections with third-party systems or to implement specialized data analysis tools. The platform also accommodates any third-party application available as a Helm Chart, Kubernetes resource, or Docker Compose, offering vast potential for customization to suit evolving industrial demands.

4.1 - Data Infrastructure

An overview of UMH’s Data Infrastructure, integrating and managing diverse data sources.

The United Manufacturing Hub’s Data Infrastructure is where all data converges. It extends the ISA95 Automation Pyramid, the usual model for data flow in factory settings. This infrastructure links each level of the traditional pyramid to the Unified Namespace (UNS), incorporating extra data sources that the typical automation pyramid doesn’t include. The data is then organized, stored, and analyzed to offer useful information for frontline workers. Afterwards, it can be sent to the a data lake or analytics platform, where business analysts can access it for deeper insights.

It comprises three primary elements:

  • Data Connectivity: This component includes an array of tools and services designed to connect various systems and sensors on the shop floor, facilitating the flow of data into the Unified Namespace.
  • Unified Namespace: Acts as the central hub for all events and messages on the shop floor, ensuring data consistency and accessibility.
  • Historian: Responsible for storing events in a time-series database, it also provides tools for data visualization, enabling both real-time and historical analytics.

Together, these elements provide a comprehensive framework for collecting, storing, and analyzing data, enhancing the operational efficiency and decision-making processes on the shop floor.

graph LR 2["`fa:fa-user **OT Professional / Shopfloor** Monitors and manages the shopfloor, including safety, automation and maintenance`"] style 2 fill:#dddddd,stroke:#9a9a9a,color:#000000 4["`**Data Warehouse/Data Lake** Stores data for analysis, on-premise or in the cloud`"] style 4 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 5["`**Automation Pyramid** Represents the layered structure of systems in manufacturing operations based on the ISA-95 model`"] style 5 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 16["`**Management Console** Configures, manages, and monitors Data and Device & Container Infrastructures in the UMH Integrated Platform`"] style 16 fill:#aaaaaa,stroke:#47a0b5,color:#000000 subgraph 50 [Data Infrastructure] style 50 fill:#ffffff,stroke:#47a0b5,color:#47a0b5 51["`**Unified Namespace** The central source of truth for all events and messages on the shop floor.`"] style 51 fill:#aaaaaa,stroke:#47a0b5,color:#000000 64["`**Historian** Stores events in a time-series database and provides visualization tools.`"] style 64 fill:#aaaaaa,stroke:#47a0b5,color:#000000 85["`**Connectivity** Includes tools and services for connecting various shop floor systems and sensors.`"] style 85 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 16-. Manages & monitors .->85 85-. Provides contextualized data .->51 85-. Provides and extracts data .->5 51-. Provides data .->4 51-. Stores data in a predefined schema .->64 5<-. Works with .-2 2-. Visualize real-time dashboards .->64

4.1.1 - Data Connectivity

Learn about the tools and services in UMH’s Data Connectivity for integrating shop floor systems.

The Data Connectivity module in the United Manufacturing Hub is designed to enable seamless integration of various data sources from the manufacturing environment into the Unified Namespace. Key components include:

  • Node-RED: A versatile programming tool that links hardware devices, APIs, and online services.
  • barcodereader: Connects to USB barcode readers, pushing data to the message broker.
  • benthos-umh: A specialized version of benthos featuring an OPC UA plugin for efficient data extraction.
  • sensorconnect: Integrates with IO-Link Masters and their sensors, relaying data to the message broker.

These tools collectively facilitate the extraction and contextualization of data from diverse sources, adhering to the ISA-95 automation pyramid model, and enhancing the Management Console’s capability to monitor and manage data flow within the UMH ecosystem.

graph LR 5["`**Automation Pyramid** Represents the layered structure of systems in manufacturing operations based on the ISA-95 model`"] style 5 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 16["`**Management Console** Configures, manages, and monitors Data and Device & Container Infrastructures in the UMH Integrated Platform`"] style 16 fill:#aaaaaa,stroke:#47a0b5,color:#000000 51["`**Unified Namespace** The central source of truth for all events and messages on the shop floor.`"] style 51 fill:#aaaaaa,stroke:#47a0b5,color:#000000 subgraph 85 [Connectivity] style 85 fill:#ffffff,stroke:#47a0b5,color:#47a0b5 86["`**Node-RED** A programming tool for wiring together hardware devices, APIs, and online services.`"] style 86 fill:#aaaaaa,stroke:#47a0b5,color:#000000 87["`**Barcode Reader** Connects to USB barcode reader devices and pushes data to the message broker.`"] style 87 fill:#aaaaaa,stroke:#47a0b5,color:#000000 88["`**Sensor Connect** Reads out IO-Link Master and their connected sensors, pushing data to the message broker.`"] style 88 fill:#aaaaaa,stroke:#47a0b5,color:#000000 89["`**benthos-umh** Customized version of benthos with an OPC UA plugin`"] style 89 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 16-. Manages & monitors .->89 89-. Provides contextualized data .->51 86-. Provides contextualized data .->51 87-. Provides contextualized data .->51 88-. Provides contextualized data .->51 89-. Extracts data via OPC UA .->5 86-. Extracts data via S7, and many more protocols .->5

4.1.1.1 - Barcodereader

This microservice is still in development and is not considered stable for production use.

Barcodereader is a microservice that reads barcodes and sends the data to the Kafka broker.

How it works

Connect a barcode scanner to the system and the microservice will read the barcodes and send the data to the Kafka broker.

What’s next

  • Read the Barcodereader reference documentation to learn more about the technical details of the Barcodereader microservice.

4.1.1.2 - Node Red

Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the Node-RED library.

How it works

Node-RED is a JavaScript-based tool that can be used to create flows that interact with the other microservices in the United Manufacturing Hub or external services.

See our guides for Node-RED to learn more about how to use it.

What’s next

  • Read the Node-RED reference documentation to learn more about the technical details of the Node-RED microservice.

4.1.1.3 - Sensorconnect

Sensorconnect automatically detects ifm gateways connected to the network and reads data from the connected IO-Link sensors.

How it works

Sensorconnect continuosly scans the given IP range for gateways, making it effectively a plug-and-play solution. Once a gateway is found, it automatically download the IODD files for the connected sensors and starts reading the data at the configured interval. Then it processes the data and sends it to the MQTT or Kafka broker, to be consumed by other microservices.

If you want to learn more about how to use sensors in your asstes, check out the retrofitting section of the UMH Learn website.

IODD files

The IODD files are used to describe the sensors connected to the gateway. They contain information about the data type, the unit of measurement, the minimum and maximum values, etc. The IODD files are downloaded automatically from IODDFinder once a sensor is found, and are stored in a Persistent Volume. If downloading from internet is not possible, for example in a closed network, you can download the IODD files manually and store them in the folder specified by the IODD_FILE_PATH environment variable.

If no IODD file is found for a sensor, the data will not be processed, but sent to the broker as-is.

What’s next

  • Read the Sensorconnect reference documentation to learn more about the technical details of the Sensorconnect microservice.

4.1.2 - Unified Namespace

Discover the Unified Namespace’s role as a central hub for shop floor data in UMH.

The Unified Namespace (UNS) within the United Manufacturing Hub is a vital module facilitating the streamlined flow and management of data. It comprises various microservices:

  • data-bridge: Bridges data between MQTT and Kafka and between multiple Kafka instances, ensuring efficient data transmission.
  • HiveMQ: An MQTT broker crucial for receiving data from IoT devices on the shop floor.
  • Redpanda (Kafka): Manages large-scale data processing and orchestrates communication between microservices.
  • Redpanda Console: Offers a graphical interface for monitoring Kafka topics and messages.

The UNS serves as a pivotal point in the UMH architecture, ensuring data from shop floor systems and sensors (gathered via the Data Connectivity module) is effectively processed and relayed to the Historian and external Data Warehouses/Data Lakes for storage and analysis.

graph LR 4["`**Data Warehouse/Data Lake** Stores data for analysis, on-premise or in the cloud`"] style 4 fill:#f4f4f4,stroke:#f4f4f4,color:#000000 64["`**Historian** Stores events in a time-series database and provides visualization tools.`"] style 64 fill:#aaaaaa,stroke:#47a0b5,color:#000000 85["`**Connectivity** Includes tools and services for connecting various shop floor systems and sensors.`"] style 85 fill:#aaaaaa,stroke:#47a0b5,color:#000000 subgraph 51 [Unified Namespace] style 51 fill:#ffffff,stroke:#47a0b5,color:#47a0b5 52["`**Redpanda (Kafka)** Handles large-scale data processing and communication between microservices.`"] style 52 fill:#aaaaaa,stroke:#47a0b5,color:#000000 53["`**HiveMQ** MQTT broker used for receiving data from IoT devices on the shop floor.`"] style 53 fill:#aaaaaa,stroke:#47a0b5,color:#000000 54["`**Redpanda Console** Provides a graphical view of topics and messages in Kafka.`"] style 54 fill:#aaaaaa,stroke:#47a0b5,color:#000000 55["`**databridge** Bridges messages between MQTT and Kafka as well as between Kafka and other Kafka instances.`"] style 55 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 54-.->52 52<-.->55 55<-.->53 55-. Provides data .->4 52-. Stores data in a predefined schema .->64 85-. Provides contextualized data .->53 85-. Provides contextualized data .->52

4.1.2.1 - Data Bridge

Data-bridge is a microservice specifically tailored to adhere to the UNS data model. It consumes topics from a message broker, translates them to the proper format and publishes them to the other message broker.

How it works

Data-bridge connects to the source broker, that can be either Kafka or MQTT, and subscribes to the topics specified in the configuration. It then processes the messages, and publishes them to the destination broker, that can be either Kafka or MQTT.

In the case where the destination broker is Kafka, messages from multiple topics can be merged into a single topic, making use of the message key to identify the source topic. For example, subscribing to a topic using a wildcard, such as umh.v1.acme.anytown..*, and a merge point of 4, will result in messages from the topics umh.v1.acme.anytown.foo.bar, umh.v1.acme.anytown.foo.baz, umh.v1.acme.anytown and umh.v1.acme.anytown.frob being merged into a single topic, umh.v1.acme.anytown, with the message key being the missing part of the topic name, in this case foo.bar, foo.baz, etc.

Here is a diagram showing the flow of messages:

graph LR source((MQTT or Kafka broker)) subgraph Messages direction TB msg1(topic: umh/v1/acme/anytown/foo/bar
value: 1) msg2(topic: umh/v1/acme/anytown/foo/baz
value: 2) msg3(topic: umh/v1/acme/anytown
value: 3) msg4(topic: umh/v1/acme/anytown/frob
value: 4) end source --> msg1 source --> msg2 source --> msg3 source --> msg4 msg1 --> bridge msg2 --> bridge msg3 --> bridge msg4 --> bridge bridge{{data-bridge
subscribes to: umh/v1/acme/anytown/#
topic merge point: 4}} subgraph Grouped messages direction TB gmsg1(topic: umh.v1.acme.anytown
key: foo.bar
value: 1) gmsg2(topic: umh.v1.acme.anytown
key: foo.baz
value: 2) gmsg3(topic: umh.v1.acme.anytown
value: 3) gmsg4(topic: umh.v1.acme.anytown
key: frob
value: 4) end bridge --> gmsg1 bridge --> gmsg2 bridge --> gmsg3 bridge --> gmsg4 dest((Kafka broker)) gmsg1 --> dest gmsg2 --> dest gmsg3 --> dest gmsg4 --> dest

The value of the message is not changed, only the topic and key are modified.

Another important feature is that it is possible to configure multiple data bridges, each with its own source and destination brokers, and each with its own set of topics to subscribe to and merge point.

The brokers can be local or remote, and, in case of MQTT, they can be secured using TLS.

What’s next

  • Read the Data Bridge reference documentation to learn more about the technical details of the data-bridge microservice.

4.1.2.2 - Kafka Broker

The Kafka broker in the United Manufacturing Hub is RedPanda, a Kafka-compatible event streaming platform. It’s used to store and process messages, in order to stream real-time data between the microservices.

How it works

RedPanda is a distributed system that is made up of a cluster of brokers, designed for maximum performance and reliability. It does not depend on external systems like ZooKeeper, as it’s shipped as a single binary.

Read more about RedPanda in the official documentation.

What’s next

  • Read the Kafka Broker reference documentation to learn more about the technical details of the Kafka broker microservice

4.1.2.3 - Kafka Console

Kafka-console uses Redpanda Console to help you manage and debug your Kafka workloads effortlessy.

With it, you can explore your Kafka topics, view messages, list the active consumers, and more.

How it works

You can access the Kafka console via its Service.

It’s automatically connected to the Kafka broker, so you can start using it right away. You can view the Kafka broker configuration in the Broker tab, and explore the topics in the Topics tab.

What’s next

  • Read the Kafka Console reference documentation to learn more about the technical details of the Kafka Console microservice.

4.1.2.4 - MQTT Broker

The MQTT broker in the United Manufacturing Hub is HiveMQ and is customized to fit the needs of the stack. It’s a core component of the stack and is used to communicate between the different microservices.

How it works

The MQTT broker is responsible for receiving MQTT messages from the different microservices and forwarding them to the MQTT Kafka bridge.

What’s next

  • Read the MQTT Broker reference documentation to learn more about the technical details of the MQTT Broker microservice.

4.1.3 - Historian

Insight into the Historian’s role in storing and visualizing data within the UMH ecosystem.

The Historian in the United Manufacturing Hub serves as a comprehensive data management and visualization system. It includes:

  • kafka-to-postgresql-v2: Archives Kafka messages adhering to the Data Model V2 schema into the database.
  • TimescaleDB: An open-source SQL database specialized in time-series data storage.
  • Grafana: A software tool for data visualization and analytics.
  • factoryinsight: An analytics tool designed for data analysis, including calculating operational efficiency metrics like OEE.
  • grafana-datasource-v2: A Grafana plugin facilitating connection to factoryinsight.
  • Redis: Utilized as an in-memory data structure store for caching purposes.

This structure ensures that data from the Unified Namespace is systematically stored, processed, and made visually accessible, providing OT professionals with real-time insights and analytics on shop floor operations.

graph LR 2["`fa:fa-user **OT Professional / Shopfloor** Monitors and manages the shopfloor, including safety, automation and maintenance`"] style 2 fill:#dddddd,stroke:#9a9a9a,color:#000000 51["`**Unified Namespace** The central source of truth for all events and messages on the shop floor.`"] style 51 fill:#aaaaaa,stroke:#47a0b5,color:#000000 subgraph 64 [Historian] style 64 fill:#ffffff,stroke:#47a0b5,color:#47a0b5 65["`**kafka-to-postgresql-v2** Stores in the database the Kafka messages that follow the Data Model V2 schema`"] style 65 fill:#aaaaaa,stroke:#47a0b5,color:#000000 66["`**TimescaleDB** An open-source time-series SQL database`"] style 66 fill:#aaaaaa,stroke:#47a0b5,color:#000000 67["`**Grafana** Visualization and analytics software`"] style 67 fill:#aaaaaa,stroke:#47a0b5,color:#000000 68["`**factoryinsight** Analytics software that allows data analysis, like OEE`"] style 68 fill:#aaaaaa,stroke:#47a0b5,color:#000000 69["`**grafana-datasource-v2** Grafana plugin to easily connect to factoryinsight`"] style 69 fill:#aaaaaa,stroke:#47a0b5,color:#000000 70["`**Redis** In-memory data structure store used for caching`"] style 70 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 65-. Stores data .->66 51-. Stores data in a predefined schema via .->65 67-. Performs SQL queries .->66 67-. Includes .->69 69-. Extracts KPIs and other high-level metrics .->68 68-. Queries data .->66 68<-.->70 65<-.->70 2-. Visualize real-time dashboards .->67

4.1.3.1 - Cache

The cache in the United Manufacturing Hub is Redis, a key-value store that is used as a cache for the other microservices.

How it works

Recently used data is stored in the cache to reduce the load on the database. All the microservices that need to access the database will first check if the data is available in the cache. If it is, it will be used, otherwise the microservice will query the database and store the result in the cache.

By default, Redis is configured to run in standalone mode, which means that it will only have one master node.

What’s next

  • Read the Cache reference documentation to learn more about the technical details of the cache microservice.

4.1.3.2 - Database

The database microservice is the central component of the United Manufacturing Hub and is based on TimescaleDB, an open-source relational database built for handling time-series data. TimescaleDB is designed to provide scalable and efficient storage, processing, and analysis of time-series data.

You can find more information on the datamodel of the database in the Data Model section, and read about the choice to use TimescaleDB in the blog article.

How it works

When deployed, the database microservice will create two databases, with the related usernames and passwords:

  • grafana: This database is used by Grafana to store the dashboards and other data.
  • factoryinsight: This database is the main database of the United Manufacturing Hub. It contains all the data that is collected by the microservices.

Then, it creates the tables based on the database schema.

If you want to learn more about how TimescaleDB works, you can read the TimescaleDB documentation.

What’s next

  • Read the Database reference documentation to learn more about the technical details of the database microservice.

4.1.3.3 - Factoryinsight

Factoryinsight is a microservice that provides a set of REST APIs to access the data from the database. It is particularly useful to calculate the Key Performance Indicators (KPIs) of the factories.

How it works

Factoryinsight exposes REST APIs to access the data from the database or calculate the KPIs. By default, it’s only accessible from the internal network of the cluster, but it can be configured to be accessible from the external network.

The APIs require authentication, that can be either a Basic Auth or a Bearer token. Both of these can be found in the Secret factoryinsight-secret.

What’s next

  • Read the Factoryinsight reference documentation to learn more about the technical details of the Factoryinsight microservice.

4.1.3.4 - Grafana

The grafana microservice is a web application that provides visualization and analytics capabilities. Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored.

It has a rich ecosystem of plugins that allow you to extend its functionality beyond the core features.

How it works

Grafana is a web application that can be accessed through a web browser. It let’s you create dashboards that can be used to visualize data from the database.

Thanks to some custom datasource plugins, Grafana can use the various APIs of the United Manufacturing Hub to query the database and display useful information.

What’s next

  • Read the Grafana reference documentation to learn more about the technical details of the grafana microservice.

4.1.3.5 - Kafka to Postgresql V2

The Kafka to PostgreSQL v2 microservice plays a crucial role in consuming and translating Kafka messages for storage in a PostgreSQL database. It aligns with the specifications outlined in the Data Model v2.

How it works

Utilizing Data Model v2, Kafka to PostgreSQL v2 is specifically configured to process messages from topics beginning with umh.v1.. Each new topic undergoes validation against Data Model v2 before message consumption begins. This ensures adherence to the defined data structure and standards.

Message payloads are scrutinized for structural validity prior to database insertion. Messages with invalid payloads are systematically rejected to maintain data integrity.

The microservice then evaluates the payload to determine the appropriate table for insertion within the PostgreSQL database. The decision is based on the data type of the payload field, adhering to the following rules:

  • Numeric data types are directed to the tag table.
  • String data types are directed to the tag_string table.

What’s next

  • Read the Kafka to Postgresql v2 reference documentation to learn more about the technical details of the Kafka to Postgresql v2 microservice.

4.1.3.6 - Umh Datasource V2

The plugin, umh-datasource-v2, is a Grafana data source plugin that allows you to fetch resources from a database and build queries for your dashboard.

How it works

  1. When creating a new panel, select umh-datasource-v2 from the Data source drop-down menu. It will then fetch the resources from the database. The loading time may depend on your internet speed.

    selectingDatasource
    selectingDatasource

  2. Select the resources in the cascade menu to build your query. DefaultArea and DefaultProductionLine are placeholders for the future implementation of the new data model.

    selectingDatasource
    selectingDatasource

  3. Only the available values for the specified work cell will be fetched from the database. You can then select which data value you want to query.

    selectingDatasource
    selectingDatasource

  4. Next you can specify how to transform the data, depending on what value you selected. For example, all the custom tags will have the aggregation options available. For example if you query a processValue:

    • Time bucket: lets you group data in a time bucket
    • Aggregates: common statistical aggregations (maximum, minimum, sum or count)
    • Handling missing values: lets you choose how missing data should be handled

    selectingDatasource
    selectingDatasource

Configuration

  1. In Grafana, navigate to the Data sources configuration panel.

    selectingConfiguration
    selectingConfiguration

  2. Select umh-v2-datasource to configure it.

    selectingConfiguration
    selectingConfiguration

  3. Configurations:

    • Base URL: the URL for the factoryinsight backend. Defaults to http://united-manufacturing-hub-factoryinsight-service/.
    • Enterprise name: previously customerID for the old datasource plugin. Defaults to factoryinsight.
    • API Key: authenticates the API calls to factoryinsight. Can be found with UMHLens by going to Secrets → factoryinsight-secret → apiKey. It should follow the format Basic xxxxxxxx.

    selectingConfiguration
    selectingConfiguration

4.2 - Device & Container Infrastructure

Understand the automated deployment and setup process in UMH’s Device & Container Infrastructure.

The Device & Container Infrastructure in the United Manufacturing Hub automates the deployment and setup of the data infrastructure in various environments. It is tailored for Edge deployments, particularly in Demilitarized Zones, to minimize latency on-premise, and also extends into the Cloud to harness its functionalities. It consists of several interconnected components:

  • Provisioning Server: Manages the initial bootstrapping of devices, including iPXE configuration and ignition file distribution.
  • Flatcar Image Server: A central repository hosting various versions of Flatcar Container Linux images, ensuring easy access and version control.
  • Customized iPXE: A specialized bootloader configured to streamline the initial boot process by fetching UMH-specific settings and configurations.
  • First and Second Stage Flatcar OS: A two-stage operating system setup where the first stage is a temporary OS used for installing the second stage, which is the final operating system equipped with specific configurations and tools.
  • Installation Script: An automated script hosted at management.umh.app, responsible for setting up and configuring the Kubernetes environment.
  • Kubernetes (k3s): A lightweight Kubernetes setup that forms the backbone of the container orchestration system.

This infrastructure ensures a streamlined, automated installation process, laying a robust foundation for the United Manufacturing Hub’s operation.

graph LR 16["`**Management Console** Configures, manages, and monitors Data and Device & Container Infrastructures in the UMH Integrated Platform`"] style 16 fill:#aaaaaa,stroke:#47a0b5,color:#000000 29["`**Provisioning Server** Handles device bootstrapping, including iPXE configuration and ignition file distribution.`"] style 29 fill:#aaaaaa,stroke:#47a0b5,color:#000000 30["`**Flatcar Image Server** Hosts Flatcar Container Linux images, providing a central repository for version management.`"] style 30 fill:#aaaaaa,stroke:#47a0b5,color:#000000 subgraph 27 [Device & Container Infrastructure] style 27 fill:#ffffff,stroke:#47a0b5,color:#47a0b5 28["`**Installation Script** This script automates the Kubernetes environment setup and configuration.`"] style 28 fill:#aaaaaa,stroke:#47a0b5,color:#000000 31["`**Kubernetes** Core of the container orchestration system, featuring a lightweight Kubernetes setup.`"] style 31 fill:#aaaaaa,stroke:#47a0b5,color:#000000 32["`**Customized iPXE** Configured bootloader for fetching UMH-specific settings, optimizing the initial boot process.`"] style 32 fill:#aaaaaa,stroke:#47a0b5,color:#000000 33["`**First Stage Flatcar OS** Transitional OS used during the installation of the permanent Flatcar OS.`"] style 33 fill:#aaaaaa,stroke:#47a0b5,color:#000000 34["`**Second Stage Flatcar OS** Final operating system, equipped with specific configurations and essential tools.`"] style 34 fill:#aaaaaa,stroke:#47a0b5,color:#000000 end 32-. "Downloads specified Flatcar version for initial boot." .->30 33-. "Retrieves image for second-stage Flatcar OS." .->30 34-. "Regularly checks for OS updates." .->30 32-. "Initiates boot-up of first-stage OS." .->33 32-. "Requests configuration and retrieves iPXE config." .->29 33-. "Installs second-stage Flatcar OS." .->34 33-. "Fetches ignition config with installation script." .->29 34-. "Executes installation script." .->28 34-. "Acquires token-specific ignition config." .->29 28-. "Installs Kubernetes (k3s) and required tools, then deploys the Data Infrastructure." .->31 28-. "Deploys Management Companion into Kubernetes." .->16

4.3 - Management Console

Delve into the functionalities and components of the UMH’s Management Console, ensuring efficient system management.

The Management Console is pivotal in configuring, managing, and monitoring the United Manufacturing Hub. It comprises a web application, a backend API and the management companion agent, all designed to ensure secure and efficient operation.

Management Console Architecture

Web Application

The client-side Web Application, available at management.umh.app enables users to register, add, and manage instances, and monitor the infrastructure within the United Manufacturing Hub. All communications between the Web Application and the user’s devices are end-to-end encrypted, ensuring complete confidentiality from the backend.

Management Companion

Deployed on each UMH instance, the Management Companion acts as an agent responsible for decrypting messages coming from the user via the Backend and executing requested actions. Responses are end-to-end encrypted as well, maintaining a secure and opaque channel to the Backend.

Backend

The Backend is the public API for the Management Console. It functions as a bridge between the Web Application and the Management Companion. Its primary role is to verify user permissions for accessing UMH instances. Importantly, the backend does not have access to the contents of the messages exchanged between the Web Application and the Management Companion, ensuring that communication remains opaque and secure.

4.4 - Legacy

This section gives an overview of the legacy microservices that can be found in older versions of the United Manufacturing Hub.

This section provides a comprehensive overview of the legacy microservices within the United Manufacturing Hub. These microservices are currently in a transitional phase, being maintained and deployed alongside newer versions of UMH as we gradually shift from Data Model v1 to v2. While these legacy components are set to be deprecated in the future, they continue to play a crucial role in ensuring smooth operations and compatibility during this transition period.

4.4.1 - Factoryinput

This microservice is still in development and is not considered stable for production use

Factoryinput provides REST endpoints for MQTT messages via HTTP requests.

This microservice is typically accessed via grafana-proxy

How it works

The factoryinput microservice provides REST endpoints for MQTT messages via HTTP requests.

The main endpoint is /api/v1/{customer}/{location}/{asset}/{value}, with a POST request method. The customer, location, asset and value are all strings. And are used to build the MQTT topic. The body of the HTTP request is used as the MQTT payload.

What’s next

  • Read the Factoryinput reference documentation to learn more about the technical details of the Factoryinput microservice.

4.4.2 - Grafana Proxy

This microservice is still in development and is not considered stable for production use

How it works

The grafana-proxy microservice serves an HTTP REST endpoint located at /api/v1/{service}/{data}. The service parameter specifies the backend service to which the request should be proxied, like factoryinput or factoryinsight. The data parameter specifies the API endpoint to forward to the backend service. The body of the HTTP request is used as the payload for the proxied request.

What’s next

  • Read the Grafana Proxy reference documentation to learn more about the technical details of the Grafana Proxy microservice.

4.4.3 - Kafka Bridge

Kafka-bridge is a microservice that connects two Kafka brokers and forwards messages between them. It is used to connect the local broker of the edge computer with the remote broker on the server.

How it works

This microservice has two ways of operation:

  • High Integrity: This mode is used for topics that are critical for the user. It is garanteed that no messages are lost. This is achieved by committing the message only after it has been successfully inserted into the database. Ususally all the topics are forwarded in this mode, except for processValue, processValueString and raw messages.
  • High Throughput: This mode is used for topics that are not critical for the user. They are forwarded as fast as possible, but it is possible that messages are lost, for example if the database struggles to keep up. Usually only the processValue, processValueString and raw messages are forwarded in this mode.

What’s next

  • Read the Kafka Bridge reference documentation to learn more about the technical details of the Kafka Bridge microservice.

4.4.4 - Kafka State Detector

This microservice is still in development and is not considered stable for production use

How it works

What’s next

4.4.5 - Kafka to Postgresql

Kafka-to-postgresql is a microservice responsible for consuming kafka messages and inserting the payload into a Postgresql database. Take a look at the Datamodel to see how the data is structured.

This microservice requires that the Kafka Topic umh.v1.kafka.newTopic exits. This will happen automatically from version 0.9.12.

How it works

By default, kafka-to-postgresql sets up two Kafka consumers, one for the High Integrity topics and one for the High Throughput topics.

The graphic below shows the program flow of the microservice.

Kafka-to-postgres-flow
Kafka-to-postgres-flow

High integrity

The High integrity topics are forwarded to the database in a synchronous way. This means that the microservice will wait for the database to respond with a non error message before committing the message to the Kafka broker. This way, the message is garanteed to be inserted into the database, even though it might take a while.

Most of the topics are forwarded in this mode.

The picture below shows the program flow of the high integrity mode.

high-integrity-data-flow
high-integrity-data-flow

High throughput

The High throughput topics are forwarded to the database in an asynchronous way. This means that the microservice will not wait for the database to respond with a non error message before committing the message to the Kafka broker. This way, the message is not garanteed to be inserted into the database, but the microservice will try to insert the message into the database as soon as possible. This mode is used for the topics that are expected to have a high throughput.

The topics that are forwarded in this mode are processValue, processValueString and all the raw topics.

What’s next

  • Read the Kafka to Postgresql reference documentation to learn more about the technical details of the Kafka to Postgresql microservice.

4.4.6 - MQTT Bridge

MQTT-bridge is a microservice that connects two MQTT brokers and forwards messages between them. It is used to connect the local broker of the edge computer with the remote broker on the server.

How it works

This microservice subscribes to topics on the local broker and publishes the messages to the remote broker, while also subscribing to topics on the remote broker and publishing the messages to the local broker.

What’s next

  • Read the MQTT Bridge reference documentation to learn more about the technical details of the MQTT Bridge microservice.

4.4.7 - MQTT Kafka Bridge

Mqtt-kafka-bridge is a microservice that acts as a bridge between MQTT brokers and Kafka brokers, transfering messages from one to the other and vice versa.

This microservice requires that the Kafka Topic umh.v1.kafka.newTopic exits. This will happen automatically from version 0.9.12.

Since version 0.9.10, it allows all raw messages, even if their content is not in a valid JSON format.

How it works

Mqtt-kafka-bridge consumes topics from a message broker, translates them to the proper format and publishes them to the other message broker.

What’s next

  • Read the MQTT Kafka Bridge reference documentation to learn more about the technical details of the MQTT Kafka Bridge microservice.

4.4.8 - MQTT Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but is enabled by default.

The IoTSensors MQTT Simulator is a microservice that simulates sensors sending data to the MQTT broker. You can read the full documentation on the

GitHub repository.

How it works

The microservice publishes messages on the topic ia/raw/development/ioTSensors/, creating a subtopic for each simulation. The subtopics are the names of the simulations, which are Temperature, Humidity, and Pressure. The values are calculated using a normal distribution with a mean and standard deviation that can be configured.

What’s next

  • Read the IoTSensors MQTT Simulator reference documentation to learn more about the technical details of the IoTSensors MQTT Simulator microservice.

4.4.9 - MQTT to Postgresql

If you landed here from Google, you probably might want to check out either the architecture of the United Manufacturing Hub or our knowledge website for more information on the general topics of IT, OT and IIoT.

This microservice is deprecated and should not be used anymore in production. Please use kafka-to-postgresql instead.

How it works

The mqtt-to-postgresql microservice subscribes to the MQTT broker and saves the values of the messages on the topic ia/# in the database.

What’s next

  • Read the MQTT to Postgresql reference documentation to learn more about the technical details of the MQTT to Postgresql microservice.

4.4.10 - OPCUA Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but is enabled by default.

How it works

The OPCUA Simulator is a microservice that simulates OPCUA devices. You can read the full documentation on the GitHub repository.

You can then connect to the simulated OPCUA server via Node-RED and read the values of the simulated devices. Learn more about how to connect to the OPCUA simulator to Node-RED in our guide.

What’s next

  • Read the OPCUA Simulator reference documentation to learn more about the technical details of the OPCUA Simulator microservice.

4.4.11 - PackML Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but it is enabled by default.

PackML MQTT Simulator is a virtual line that interfaces using PackML implemented over MQTT. It implements the following PackML State model and communicates over MQTT topics as defined by environmental variables. The simulator can run with either a basic MQTT topic structure or SparkPlugB.

PackML StateModel
PackML StateModel

How it works

You can read the full documentation on the GitHub repository.

What’s next

  • Read the PackML Simulator reference documentation to learn more about the technical details of the PackML Simulator microservice.

4.4.12 - Tulip Connector

This microservice is still in development and is not considered stable for production use.

The tulip-connector microservice enables communication with the United Manufacturing Hub by exposing internal APIs, like factoryinsight, to the internet. With this REST endpoint, users can access data stored in the UMH and seamlessly integrate Tulip with a Unified Namespace and on-premise Historian. Furthermore, the tulip-connector can be customized to meet specific customer requirements, including integration with an on-premise MES system.

How it works

The tulip-connector acts as a proxy between the internet and the UMH. It exposes an endpoint to forward requests to the UMH and returns the response.

What’s next

  • Read the Tulip Connector reference documentation to learn more about the technical details of the Tulip Connector microservice.

4.4.13 - Grafana Plugins

This section contains the overview of the custom Grafana plugins that can be used to access the United Manufacturing Hub.

4.4.13.1 - Umh Datasource

This page contains the technical documentation of the plugin umh-datasource, which allows for easy data extraction from factoryinsight.

We are no longer maintaining this microservice. Use instead our new microservice datasource-v2 for data extraction from factoryinsight.

The umh datasource is a Grafana 8.X compatible plugin, that allows you to fetch resources from a database and build queries for your dashboard.

How it works

  1. When creating a new panel, select umh-datasource from the Data source drop-down menu. It will then fetch the resources from the database. The loading time may depend on your internet speed.

    selectingDatasource
    selectingDatasource

  2. Select your query parameters Location, Asset and Value to build your query.

    selectingDatasource
    selectingDatasource

Configuration

  1. In Grafana, navigate to the Data sources configuration panel.

    selectingConfiguration
    selectingConfiguration

  2. Select umh-datasource to configure it.

    selectingConfiguration
    selectingConfiguration

  3. Configurations:

    • Base URL: the URL for the factoryinsight backend. Defaults to http://united-manufacturing-hub-factoryinsight-service/.
    • Enterprise name: previously customerID for the old datasource plugin. Defaults to factoryinsight.
    • API Key: authenticates the API calls to factoryinsight. Can be found with UMHLens by going to Secrets → factoryinsight-secret → apiKey. It should follow the format Basic xxxxxxxx.

    selectingConfiguration
    selectingConfiguration

4.4.13.2 - Factoryinput Panel

This page contains the technical documentation of the plugin factoryinput-panel, which allows for easy execution of MQTT messages inside the UMH stack from a Grafana panel.

This plugin is still in development and is not considered stable for production use

Requirements

  • A United Manufacturing Hub stack
  • External IP or URL to the grafana-proxy
    • In most cases it is the same IP address as your Grafana dashboard.

Getting started

For development, the steps to build the plugin from source are described here.

  1. Go to united-manufacturing-hub/grafana-plugins/umh-factoryinput-panel
  2. Install dependencies.
yarn install
  1. Build plugin in development mode or run in watch mode.
yarn dev
  1. Build plugin in production mode (not recommended due to Issue 32336).
yarn build
  1. Move the resulting dis folder in your Grafana plugins directory.
  • Windows: C:\Program Files\GrafanaLabs\grafana\data\plugins
  • Linux: /var/lib/grafana/plugins

  1. Rename the folder to umh-factoryinput-panel.

  2. Enable the enable development mode to load unsigned plugins.

  3. restart your Grafana service.

Technical Information

Below you will find a schematic of this flow, through our stack.

5 - Production Guide

This section contains information about how to use the stack in a production environment.

5.1 - Installation

This section contains guides on how to install the United Manufacturing Hub.

Learn how to install the United Manufacturing Hub using completely Free and Open Source Software.

5.1.1 - Flatcar Installation

This page describes how to deploy the United Manufacturing Hub on Flatcar Linux.

Here is a step-by-step guide on how to deploy the United Manufacturing Hub on Flatcar Linux, a Linux distribution designed for container workloads with high security and low maintenance. This will leverage the UMH Device and Container Infrastructure.

The system can be installed either bare metal or in a virtual machine.

Before you begin

Ensure your system meets these minimum requirements:

  • 4-core CPU
  • 8 GB system RAM
  • 32 GB available disk space
  • Internet access

You will also need the latest version of the iPXE boot image, suitable for your system:

For bare metal installations, flash the image to a USB stick with at least 4 GB of storage. Our guide on flashing an operating system to a USB stick can assist you.

For virtual machines, ensure UEFI boot is enabled when creating the VM.

Lastly, ensure you are on the same network as the device for SSH access post-installation.

System Preparation and Booting from iPXE

Identify the drive for Flatcar Linux installation. For virtual machines, this is typically sda. For bare metal, the drive depends on your physical storage. The troubleshooting section can help identify the correct drive.

Boot your device from the iPXE image. Consult your device or hypervisor documentation for booting instructions.

You can find a comprehensive guide on how to configure a virtual machine in Proxmox for installing Flatcar Linux on the Learning Hub.

Installation

At the first prompt, read and accept the license to proceed.

Read and Accept the License
Read and Accept the License

Next, configure your network settings. Select DHCP if uncertain.

Network Settings
Network Settings

The connection will be tested next. If it fails, revisit the network settings.

Ensure your device has internet access and no firewalls are blocking the connection.

Then, select the drive for Flatcar Linux installation.

Select the Drive
Select the Drive

A summary of the installation will appear. Check that everything is correct and confirm to start the process.

Summary
Summary

Shortly after, you’ll see a green command line core@flatcar-0-install. Remove the USB stick or the CD drive from the VM. The system will continue processing.

Flatcar Install Step 0
Flatcar Install Step 0

The installation will complete after a few minutes, and the system will reboot.

When you see the green core@flatcar-1-umh login prompt, the installation is complete, and the device’s IP address will be displayed.

Installation time varies based on network speed and system performance.

Connect to the Device

With the system installed, access it via SSH.

For Windows 11 users, the default Windows Terminal is recommended. For other OS users, try MobaXTerm.

To do so, open you terminal of choice. We recommend the default Windows Terminal, or MobaXTerm if you are not on Windows 11.

Connect to the device using this command, substituting <ip-address> with your device’s IP address:

ssh core@<ip-address>

When prompted, enter the default password for the core user: umh.

Troubleshooting

The Installation Stops at the First Green Login Prompt

If the installation halts at the first green login prompt, check the installation status with:

systemctl status installer

A typical response for an ongoing installation will look like this:

● installer.service - Flatcar Linux Installer
     Loaded: loaded (/usr/lib/systemd/system/installer.service; static; vendor preset: enabled)
     Active: active (running) since Wed 2021-05-12 14:00:00 UTC; 1min 30s ago

If the status differs, the installation may have failed. Review the logs to identify the issue.

Unsure Which Drive to Select

To determine the correct drive, refer to your device’s manual:

  • SATA drives (HDD or SSD): Typically labeled as sda.
  • NVMe drives: Usually labeled as nvm0n1.

For further verification, boot any Linux distribution on your device and execute:

lsblk

The output, resembling the following, will help identify the drive:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 223.6G  0 disk
├─sda1   8:1    0   512M  0 part /boot
└─sda2   8:2    0 223.1G  0 part /
sdb      8:0    0  31.8G  0 disk
└─sdb1   8:1    0  31.8G  0 part /mnt/usb

In most cases, the correct drive is the first listed or the one not matching the USB stick size.

No Resources in the Cluster

If you can access the cluster but see no resources, SSH into the edge device and check the cluster status:

systemctl status k3s

If the status is not active (running), the cluster isn’t operational. Restart it with:

sudo systemctl restart k3s

If the cluster is active or restarting doesn’t resolve the issue, inspect the installation logs:

systemctl status umh-install
systemctl status helm-install

Persistent errors may necessitate a system reinstallation.

I can’t SSH into the virtual machine

Ensure that your computer is on the same network as the virtual machine, with no firewalls or VPNs blocking the connection.

What’s next

  • You can follow the Getting Started guide to get familiar with the UMH stack.
  • If you already know your way around the United Manufacturing Hub, you can follow the Administration guides to configure the stack for production.

5.2 - Upgrading

This section contains the upgrading guides for the different versions the United Manufacturing Hub.

The United Manufacturing Hub is a continuously evolving product. This means that new features and bug fixes are added to the product on a regular basis. This section contains the upgrading guides for the different versions the United Manufacturing Hub.

The upgrading process is done by upgrading the Helm chart.

5.2.1 - Upgrade to v0.9.15

This page describes how to upgrade the United Manufacturing Hub to version 0.9.15

This page describes how to upgrade the United Manufacturing Hub to version 0.9.15. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-opcuasimulator-deployment
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-grafanaproxy
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-kafka
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect
    • united-manufacturing-hub-mqttbridge
  4. Open the Network tab.
  5. From the Services section, delete the following services:
    • united-manufacturing-hub-kafka

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. You can also change the values of the Helm chart, if needed. If you want to activate the new databridge you need to add & edit the following section

    _000_commonConfig:      
   ...
   datamodel_v2:
   enabled: true
   bridges:
   - mode: mqtt-kafka
     brokerA: united-manufacturing-hub-mqtt:1883 # The flow is always from A->B, for omni-directional flow, setup a 2nd bridge with reversed broker setup
     brokerB: united-manufacturing-hub-kafka:9092
     topic: umh.v1..*              # accept mqtt or kafka topic format. after the topic seprator, you can use # for mqtt wildcard, or .* for kafka wildcard
     topicMergePoint: 5            # This is a new feature of our datamodel, which splits topics in topic and key (only in Kafka), preventing having lots of topics
     partitions: 6                 # optional: number of partitions for the new kafka topic. default: 6
     replicationFactor: 1          # optional: replication factor for the new kafka topic. default: 1
   ...

    You can also enable the new container registry by changing the values in the image or image.repository fields from unitedmanufacturinghub/<image-name> to ghcr.io/united-manufacturing-hub/<image-name>.

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.2 - Upgrade to v0.9.14

This page describes how to upgrade the United Manufacturing Hub to version 0.9.14

This page describes how to upgrade the United Manufacturing Hub to version 0.9.14. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-opcuasimulator-deployment
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-grafanaproxy
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-kafka
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect
    • united-manufacturing-hub-mqttbridge
  4. Open the Network tab.
  5. From the Services section, delete the following services:
    • united-manufacturing-hub-kafka

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. You can also change the values of the Helm chart, if needed. For example, if you want to apply the new tweaks to the resources in order to avoid the Out Of Memory crash of the MQTT Broker, you can change the following values:

    iotsensorsmqtt:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
grafanaproxy:
 resources:
   requests:
     cpu: 100m
   limits:
     cpu: 300m
kafkatopostgresql:
  resources:
    requests:
      memory: 150Mi
    limits:
      memory: 300Mi
opcuasimulator:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
packmlmqttsimulator:
  resources:
    requests:
      cpu: 10m
      memory: 20Mi
    limits:
      cpu: 30m
      memory: 50Mi
tulipconnector:
  resources:
    limits:
      cpu: 30m
      memory: 50Mi
    requests:
      cpu: 10m
      memory: 20Mi
redis:
  master:
    resources:
      limits:
        cpu: 100m
        memory: 100Mi
      requests:
        cpu: 50m
        memory: 50Mi
mqtt_broker:
  resources:
    limits:
      cpu: 700m
      memory: 1700Mi
    requests:
      cpu: 300m
      memory: 1000Mi

    You can also enable the new container registry by changing the values in the image or image.repository fields from unitedmanufacturinghub/<image-name> to ghcr.io/united-manufacturing-hub/<image-name>.

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.3 - Upgrade to v0.9.13

This page describes how to upgrade the United Manufacturing Hub to version 0.9.13

This page describes how to upgrade the United Manufacturing Hub to version 0.9.13. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed.
  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.4 - Upgrade to v0.9.12

This page describes how to upgrade the United Manufacturing Hub to version 0.9.12

This page describes how to upgrade the United Manufacturing Hub to version 0.9.12. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Backup RBAC configuration for MQTT Broker

This step is only needed if you enabled RBAC for the MQTT Broker and changed the default password. If you did not change the default password, you can skip this step.

  1. Navigate to Config > ConfigMaps.
  2. Select the united-manufacturing-hub-hivemqce-extension ConfigMap.
  3. Copy the content of credentials.xml and save it in a safe place.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Remove MQTT Broker extension PVC

In this version we reduced the size of the MQTT Broker extension PVC. To do so, we need to delete the old PVC and create a new one. This process will set the credentials of the MQTT Broker to the default ones. If you changed the default password, you can restore them after the upgrade.

  1. Navigate to Storage > Persistent Volume Claims.
  2. Select the united-manufacturing-hub-hivemqce-claim-extensions PVC and click Delete.

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. There are some incompatible changes in this version. To avoid errors, you need to change the following values:

    • Remove property console.console.config.kafka.tls.passphrase:

      console:
  console:
    config:
      kafka:
        tls:
          passphrase: "" # <- remove this line

    • console.extraContainers: remove the property and its content.

      console:
  extraContainers: {} # <- remove this line

    • console.extraEnv: remove the property and its content.

      console:
  extraEnv: "" # <- remove this line

    • console.extraEnvFrom: remove the property and its content.

      console:
  extraEnvFrom: ""  # <- remove this line

    • console.extraVolumeMounts: remove the |- characters right after the property name. It should look like this:

      console:
  extraVolumeMounts: # <- remove the `|-` characters in this line
    - name: united-manufacturing-hub-kowl-certificates
      mountPath: /SSL_certs/kafka
      readOnly: true

    • console.extraVolumes: remove the |- characters right after the property name. It should look like this:

      console:
  extraVolumes: # <- remove the `|-` characters in this line
    - name: united-manufacturing-hub-kowl-certificates
      secret:
        secretName: united-manufacturing-hub-kowl-secrets

    • Change the console.service property to the following:

      console:
  service:
    type: LoadBalancer
    port: 8090
    targetPort: 8080

    • Change the Redis URI in factoryinsight.redis:

      factoryinsight:
  redis:
    URI: united-manufacturing-hub-redis-headless:6379

    • Set the following values in the kafka section to true, or add them if they are missing:

      kafka:
  externalAccess:
    autoDiscovery:
      enabled: true
    enabled: true
  rbac:
    create: true

    • Change redis.architecture to standalone:

      redis:
  architecture: standalone

    • redis.sentinel: remove the property and its content.

      redis:
  sentinel: {} # <- remove all the content of this section

    • Remove the property redis.master.command:

      redis:
  master:
  command: /run.sh # <- remove this line

    • timescaledb-single.fullWalPrevention: remove the property and its content.

      timescaledb-single:
  fullWalPrevention:              # <- remove this line
    checkFrequency: 30            # <- remove this line
    enabled: false                # <- remove this line
    thresholds:                   # <- remove this line
      readOnlyFreeMB: 64          # <- remove this line
      readOnlyFreePercent: 5      # <- remove this line
      readWriteFreeMB: 128        # <- remove this line
      readWriteFreePercent: 8     # <- remove this line

    • timescaledb-single.loadBalancer: remove the property and its content.

      timescaledb-single:
  loadBalancer:          # <- remove this line
    annotations:         # <- remove this line
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "4000" # <- remove this line
    enabled: true        # <- remove this line
    port: 5432           # <- remove this line

    • timescaledb-single.replicaLoadBalancer: remove the property and its content.

      timescaledb-single:
  replicaLoadBalancer:
    annotations:         # <- remove this line
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "4000" # <- remove this line
    enabled: false       # <- remove this line
    port: 5432           # <- remove this line

    • timescaledb-single.secretNames: remove the property and its content.

      timescaledb-single:
  secretNames: {} # <- remove this line

    • timescaledb-single.unsafe: remove the property and its content.

      timescaledb-single:
  unsafe: false # <- remove this line

    • Change the value of the timescaledb-single.service.primary.type property to LoadBalancer:

      timescaledb-single:
  service:
    primary:
      type: LoadBalancer

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.5 - Upgrade to v0.9.11

This page describes how to upgrade the United Manufacturing Hub to version 0.9.11

This page describes how to upgrade the United Manufacturing Hub to version 0.9.11. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed.
  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.6 - Upgrade to v0.9.10

This page describes how to upgrade the United Manufacturing Hub to version 0.9.10

This page describes how to upgrade the United Manufacturing Hub to version 0.9.10. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Grafana plugins

In this release, the Grafana version has been updated from 8.5.9 to 9.3.1. Check the release notes for further information about the changes.

Additionally, the way default plugins are installed has changed. Unfortunatly, it is necesary to manually install all the plugins that were previously installed.

If you didn’t install any plugin other than the default ones, you can skip this section.

Follow these steps to see the list of plugins installed in your cluster:

  1. Open the browser and go to the Grafana dashboard.

  2. Navigate to the Configuration > Plugins tab.

  3. Select the Installed filter.

    Show installed grafana plugins
    Show installed grafana plugins

  4. Write down all the plugins that you manually installed. You can recognize them by not having the Core tag.

    Image of core and signed plugins
    Image of core and signed plugins

    The following ones are installed by default, therefore you can skip them:

    • ACE.SVG by Andrew Rodgers
    • Button Panel by UMH Systems Gmbh
    • Button Panel by CloudSpout LLC
    • Discrete by Natel Energy
    • Dynamic Text by Marcus Olsson
    • FlowCharting by agent
    • Pareto Chart by isaozler
    • Pie Chart (old) by Grafana Labs
    • Timepicker Buttons Panel by williamvenner
    • UMH Datasource by UMH Systems Gmbh
    • Untimely by factry
    • Worldmap Panel by Grafana Labs

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-grafana
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. You can also change the values of the Helm chart, if needed.

    • In the grafana section, find the extraInitContainers field and change its value to the following:

          - image: unitedmanufacturinghub/grafana-umh:1.1.2
      name: init-plugins
      imagePullPolicy: IfNotPresent
      command: ['sh', '-c', 'cp -r /plugins /var/lib/grafana/']
      volumeMounts:
        - name: storage
          mountPath: /var/lib/grafana

    • Make these changes in the kafka section:

      • Set the value of the heapOpts field to -Xmx2048m -Xms2048m.

      • Replace the content of the resources section with the following:

            limits:
      cpu: 1000m
      memory: 4Gi
    requests:
      cpu: 100m
      memory: 2560Mi

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Afterwards, you can reinstall the additional Grafana plugins.

Replace VerneMQ with HiveMQ

In this upgrade we switched from using VerneMQ to HiveMQ as our MQTT Broker (you can read the blog article about it).

While this process is fully backwards compatible, we suggest to update NodeRed flows and any other additional service that uses MQTT, to use the new service broker called united-manufacturing-hub-mqtt. The old united-manufacturing-hub-vernemq is still functional and, despite the name, also points to HiveMQ, but in future upgrades will be removed.

Additionally, for production environments, we recommend to enable RBAC for the MQTT Broker.

Please double-check if all of your services can connect to the new MQTT broker. It might be needed for them to be restarted, so that they can resolve the DNS name and get the new IP. Also, it can happen with tools like chirpstack, that you need to specify the client-id as the automatically generated ID worked with VerneMQ, but is now declined by HiveMQ.

Troubleshooting

Some microservices can’t connect to the new MQTT broker

If you are using the united-manufacturing-hub-mqtt service, but some microservice can’t connect to it, restarting the microservice might solve the issue. To do so, you can delete the Pod of the microservice and let Kubernetes recreate it.

ChirpStack can’t connect to the new MQTT broker

ChirpStack uses a generated client-id to connect to the MQTT broker. This client-id is not accepted by HiveMQ. To solve this issue, you can set the client_id field in the integration.mqtt section of the chirpstack configuration file to a fixed value:

[integration]
...
  [integration.mqtt]
  client_id="chirpstack"

5.2.7 - Upgrade to v0.9.9

This page describes how to upgrade the United Manufacturing Hub to version 0.9.9

This page describes how to upgrade the United Manufacturing Hub to version 0.9.9. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed. In the grafana section, find the extraInitContainers field and change the value of the image field to unitedmanufacturinghub/grafana-plugin-extractor:0.1.4.
  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.8 - Upgrade to v0.9.8

This page describes how to upgrade the United Manufacturing Hub to version 0.9.8

This page describes how to upgrade the United Manufacturing Hub to version 0.9.8. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed.
  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.9 - Upgrade to v0.9.7

This page describes how to upgrade the United Manufacturing Hub to version 0.9.7

This page describes how to upgrade the United Manufacturing Hub to version 0.9.7. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed.

    • Make these changes in the grafana section:

      • Replace the content of datasources with the following:

            datasources.yaml:
      apiVersion: 1
      datasources:
      - access: proxy
        editable: false
        isDefault: true
        jsonData:
          apiKey: $FACTORYINSIGHT_PASSWORD
          apiKeyConfigured: true
          customerId: $FACTORYINSIGHT_CUSTOMERID
          serverURL: http://united-manufacturing-hub-factoryinsight-service/
        name: umh-datasource
        orgId: 1
        type: umh-datasource
        url: http://united-manufacturing-hub-factoryinsight-service/
        version: 1
      - access: proxy
        editable: false
        isDefault: false
        jsonData:
          apiKey: $FACTORYINSIGHT_PASSWORD
          apiKeyConfigured: true
          baseURL: http://united-manufacturing-hub-factoryinsight-service/
          customerID: $FACTORYINSIGHT_CUSTOMERID
        name: umh-v2-datasource
        orgId: 1
        type: umh-v2-datasource
        url: http://united-manufacturing-hub-factoryinsight-service/
        version: 1

      • Replace the content of env with the following:

            GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: umh-datasource,umh-factoryinput-panel,umh-v2-datasource

      • Replace the content of extraInitContainers with the following:

          - name: init-umh-datasource
    image: unitedmanufacturinghub/grafana-plugin-extractor:0.1.3
    volumeMounts:
    - name: storage
      mountPath: /var/lib/grafana
    imagePullPolicy: IfNotPresent

    • In the timescaledb-single section, make sure that the image.tag field is set to pg13.8-ts2.8.0-p1.

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Change Factoryinsight API version

The Factoryinsight API version has changed from v1 to v2. To make sure that you are using the new version, click on any Factoryinsight Pod and check that the VERSION environment variable is set to 2.

If it’s not, follow these steps:

  1. Navigate to the Workloads > Deployments tab.
  2. Select the united-manufacturing-hub-factoryinsight-deployment deployment.

  3. Click the Edit button to open the deployment’s configuration.

    Lens deployment Edit
    Lens deployment Edit

  4. Find the spec.template.spec.containers[0].env field.
  5. Set the value field of the VERSION variable to 2.

5.2.10 - Upgrade to v0.9.6

This page describes how to upgrade the United Manufacturing Hub to version 0.9.6

This page describes how to upgrade the United Manufacturing Hub to version 0.9.6. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Add new index to the database

In this version, a new index has been added to the processValueTabe table, allowing to speed up the queries.

Open a shell in the database

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Create the index

Execute the following query:

CREATE INDEX ON processvaluetable(valuename, asset_id) WITH (timescaledb.transaction_per_chunk);
REINDEX TABLE processvaluetable;

This command could take a while to complete, especially on larger tables.

Type exit to close the shell.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.
  2. Select the united-manufacturing-hub release and click Upgrade.
  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.
  4. You can also change the values of the Helm chart, if needed.
  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

5.2.11 - Upgrade to v0.9.5

This page describes how to upgrade the United Manufacturing Hub to version 0.9.5

This page describes how to upgrade the United Manufacturing Hub to version 0.9.5. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Alter ordertable constraint

In this version, one of the constraints of the ordertable table has been modified.

Make sure to backup the database before exectuing the following steps.

Open a shell in the database

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Alter the table

  1. Check for possible conflicts in the ordertable table:

    SELECT order_name, asset_id, count(*) FROM ordertable GROUP BY order_name, asset_id HAVING count(*) > 1;

    If the result is empty, you can skip the next step.

  2. Delete the duplicates:

    DELETE FROM ordertable ox USING (
     SELECT MIN(CTID) as ctid, order_name, asset_id
     FROM ordertable
     GROUP BY order_name, asset_id HAVING count(*) > 1
     ) b
WHERE ox.order_name = b.order_name AND ox.asset_id = b.asset_id
AND ox.CTID <> b.ctid;

    If the data cannot be deleted, you have to manually update each duplicate order_names to a unique value.

  3. Get the name of the constraint:

    SELECT conname FROM pg_constraint WHERE conrelid = 'ordertable'::regclass AND contype = 'u';

  4. Drop the constraint:

    ALTER TABLE ordertable DROP CONSTRAINT ordertable_asset_id_order_id_key;

  5. Add the new constraint:

    ALTER TABLE ordertable ADD CONSTRAINT ordertable_asset_id_order_name_key UNIQUE (asset_id, order_name);

Now you can close the shell by typing exit and continue with the upgrade process.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. You can also change the values of the Helm chart, if needed.

    • Enable the startup probe for the Kafka Broker by adding the following into the kafka section:

      startupProbe:
  enabled: true
  failureThreshold: 600
  periodSeconds: 10
  timeoutSeconds: 10

  5. Click Upgrade.

The upgrade process can take a few minutes. The upgrade is complete when the Status field of the release is Deployed.

Changes to the messages

Some messages have been modified in this version. You need to update some payolads in your Node-RED flows.

  • modifyState:
    • start_time_stamp has been renamed to timestamp_ms
    • end_time_stamp has been renamed to timestamp_ms_end
  • modifyProducedPieces:
    • start_time_stamp has been renamed to timestamp_ms
    • end_time_stamp has been renamed to timestamp_ms_end
  • deleteShiftByAssetIdAndBeginTimestamp and deleteShiftById have been removed. Use the deleteShift message instead.

5.2.12 - Upgrade to v0.9.4

This page describes how to upgrade the United Manufacturing Hub to version 0.9.4

This page describes how to upgrade the United Manufacturing Hub to version 0.9.4. Before upgrading, remember to backup the database, Node-RED flows, and your cluster configuration.

Add Helm repo in UMHLens / OpenLens

Check if the UMH Helm repository is added in UMHLens / OpenLens. To do so, from the top-left menu, select FIle > Preferences (or press CTRL + ,). Click on the Kubernetes tab and check if the Helm Chart section contains the https://repo.umh.app repository.

If it doesn’t, click the Add Custom Helm Repo button and fill in the following values:

Then click Add.

Clear Workloads

Some workloads need to be deleted before upgrading. This process does not delete any data, but it will cause downtime. If a workload is missing, it means that it was not enabled in your cluster, therefore you can skip it.

To delete a resource, you can select it using the box on the left of the resource name and click the - button on the bottom right corner.

  1. Open the Workloads tab.
  2. From the Deployment section, delete the following deployments:
    • united-manufacturing-hub-barcodereader
    • united-manufacturing-hub-factoryinsight-deployment
    • united-manufacturing-hub-kafkatopostgresql
    • united-manufacturing-hub-mqttkafkabridge
    • united-manufacturing-hub-iotsensorsmqtt
    • united-manufacturing-hub-opcuasimulator-deployment
  3. From the StatefulSet section, delete the following statefulsets:
    • united-manufacturing-hub-mqttbridge
    • united-manufacturing-hub-hivemqce
    • united-manufacturing-hub-nodered
    • united-manufacturing-hub-sensorconnect

Upgrade Helm Chart

Now everything is ready to upgrade the Helm chart.

  1. Navigate to the Helm > Releases tab.

  2. Select the united-manufacturing-hub release and click Upgrade.

  3. In the Helm Upgrade window, make sure that the Upgrade version field contains the version you want to upgrade to.

  4. You can also change the values of the Helm chart, if needed.

    • If you have enabled the Kafka Bridge, find the section _000_commonConfig.kafkaBridge.topicmap and set the value to the following:

      - bidirectional: false
  name: HighIntegrity
  send_direction: to_remote
  topic: ^ia\.(([^r.](\d|-|\w)*)|(r[b-z](\d|-|\w)*)|(ra[^w]))\.(\d|-|\w|_)+\.(\d|-|\w|_)+\.((addMaintenanceActivity)|(addOrder)|(addParentToChild)|(addProduct)|(addShift)|(count)|(deleteShiftByAssetIdAndBeginTimestamp)|(deleteShiftById)|(endOrder)|(modifyProducedPieces)|(modifyState)|(productTag)|(productTagString)|(recommendation)|(scrapCount)|(startOrder)|(state)|(uniqueProduct)|(scrapUniqueProduct))$
- bidirectional: false
  name: HighThroughput
  send_direction: to_remote
  topic: ^ia\.(([^r.](\d|-|\w)*)|(r[b-z](\d|-|\w)*)|(ra[^w]))\.(\d|-|\w|_)+\.(\d|-|\w|_)+\.(process[V|v]alue).*$

      For more information, see the Kafka Bridge configuration

    • If you have enabled Barcodereader, find the barcodereader section and set the following values, adding the missing ones and updating the already existing ones:

      enabled: false
image:
  pullPolicy: IfNotPresent
resources:
  requests:
    cpu: "2m"
    memory: "30Mi"
  limits:
    cpu: "10m"
    memory: "60Mi"
scanOnly: false # Debug mode, will not send data to kafka

  5. Click Upgrade.

The upgrade process can take a few minutes. The process is complete when the Status field of the release is Deployed.

5.3 - Administration

This section describes how to manage and configure the United Manufacturing Hub cluster.

In this section, you will find information about how to manage and configure the United Manufacturing Hub cluster, from customizing the cluster to access the different services.

5.3.1 - Access the Database

This page describes how to access the United Manufacturing Hub database to perform SQL operations using a database client, the CLI or Grafana.

There are multiple ways to access the database. If you want to just visualize data, then using Grafana or a database client is the easiest way. If you need to also perform SQL commands, then using a database client or the CLI are the best options.

Generally, using a database client gives you the most flexibility, since you can both visualize the data and manipulate the database. However, it requires you to install a database client on your machine.

Using the CLI gives you more control over the database, but it requires you to have a good understanding of SQL.

Grafana, on the other hand, is for visualizing data. It is a good option if you just want to see the data in a dashboard and don’t need to manupulate it.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Get the database credentials

If you are not using the CLI, you need to know the database credentials. You can find them in the timescale-post-init-pw Secret. By default, the username is factoryinsight and the password is changeme.

...
ALTER USER factoryinsight WITH PASSWORD 'changeme';
...

Access the database using a database client

There are many database clients that you can use to access the database. Here’s a list of some of the most popular database clients:

Database clients
NameFree or PaidPlatforms
pgAdminFreeWindows, macOS, Linux
DataGripPaidWindows, macOS, Linux
DBeaverBothWindows, macOS, Linux

For the sake of this tutorial, pgAdmin will be used as an example, but other clients have similar functionality. Refer to the specific client documentation for more information.

Forward the database port to your local machine

  1. From the Pods section in UMHLens / OpenLens, find the united-manufacturing-hub-timescaledb-0 Pod.
  2. In the Pod Details window, click the Forward button next to the postgresql:5432/TCP port.
  3. Enter a port number, such as 5432, and click Start. You can disable the Open in browser option if you don’t want to open the port in your browser.

Using pgAdmin

You can use pgAdmin to access the database. To do so, you need to install the pgAdmin client on your machine. For more information, see the pgAdmin documentation.

  1. Once you have installed the client, you can add a new server from the main window.

    pgAdmin main window
    pgAdmin main window

  2. In the General tab, give the server a meaningful name. In the Connection tab, enter the database credentials:

    • The Host name/address is localhost.
    • The Port is the port you forwarded.
    • The Maintenance database is postgres.
    • The Username and Password are the ones you found in the Secret.

  3. Click Save to save the server.

    pgAdmin connection window
    pgAdmin connection window

You can now connect to the database by double-clicking the server.

Use the side menu to navigate through the server. The tables are listed under the Schemas > public > Tables section of the factoryinsight database.

Refer to the pgAdmin documentation for more information on how to use the client to perform database operations.

Access the database using the command line interface

You can access the database from the command line using the psql command directly from the united-manufacturing-hub-timescaledb-0 Pod.

You will not need credentials to access the database from the Pod’s CLI.

Open a shell in the database Pod

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Perform SQL commands

Once you have a shell in the database, you can perform SQL commands.

  1. For example, to create an index on the processValueTable:

    CREATE INDEX ON processvaluetable (valuename);

  2. When you are done, exit the postgres shell:

     exit

Access the database using Grafana

You can use Grafana to visualize data from the database.

Add PostgreSQL as a data source

  1. Open the Grafana dashboard in your browser.

  2. From the Configuration (gear) icon, select Data Sources.

  3. Click Add data source and select PostgreSQL.

  4. Configure the connection to the database:

    • The Host is united-manufacturing-hub.united-manufacturing-hub.svc.cluster.local:5432.
    • The Database is factoryinsight.
    • The User and Password are the ones you found in the Secret.
    • Set TLS/SSL Mode to require.
    • Enable TimescaleDB.

    Everything else can be left as the default.

    Grafana PostgreSQL data source
    Grafana PostgreSQL data source

  5. Click Save & Test to save the data source.

  6. Now click on Explore to start querying the database.

  7. You can also create dashboards using the newly created data source.

What’s next

5.3.2 - Access Services From Within the Cluster

This page describes how to access services from within the cluster.

All the services deployed in the cluster are visible to each other. That makes it easy to connect them together.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Connect to a service from another service

To connect to a service from another service, you can use the service name as the host name.

To get a list of available services and related ports you can open UMHLens / OpenLens and go to Network > Services.

All of them are available from within the cluster. The ones of type LoadBalancer are also available from outside the cluster using the node IP.

Example

The most common use case is to connect to the MQTT Broker from Node-RED.

To do that, when you create the MQTT node, you can use the service name united-manufacturing-hub-mqtt as the host name and one the ports listed in the Ports column.

The MQTT service name has changed since version 0.9.10. If you are using an older version, use united-manufacturing-hub-vernemq instead of united-manufacturing-hub-mqtt.

What’s next

5.3.3 - Access Services Outside the Cluster

This page describe how to access services from outside the cluster.

Some of the microservices in the United Manufacturing Hub are exposed outside the cluster with a LoadBalancer service. A LoadBalancer is a service that exposes a set of Pods on the same network as the cluster, but not necessarily to the entire internet. The LoadBalancer service provides a single IP address that can be used to access the Pods.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Accessing the services

The LoadBalancer service provides a single IP address that can be used to access the Pods. To find the IP address, open UMHLens / OpenLens and navigate to Network > Services. The IP address is listed in the External IP column.

To access the services, use the IP address and the port number of the service, e.g. http://192.168.1.100:8080.

If you installed the United Manufacturing Hub on your local machine, either using the Management Console or the command line, the services are accessible at localhost:<port-number>.

Services with LoadBalancer by default

The following services are exposed outside the cluster with a LoadBalancer service by default:

To access Node-RED, you need to use the /nodered path, for example http://192.168.1.100:1880/nodered.

Services with NodePort by default

The Kafka Broker uses the service type NodePort by default.

Follow these steps to access the Kafka Broker outside the cluster:

  1. Open UMHLens / OpenLens and navigate to Network > Services.

  2. Select the Service united-manufacturing-hub-kafka-external and click the Edit button.

  3. Scroll down to the line that shows:

      type: NodePort

    Replace this with:

      type: LoadBalancer

  4. Click Save to apply the changes.

Access the Kafka Broker at port 9094.

Services with ClusterIP

Some of the microservices in the United Manufacturing Hub are exposed via a ClusterIP service. That means that they are only accessible from within the cluster itself. To access them from outside the cluster, you need to create a LoadBalancer service.

Create a LoadBalancer service

If you are looking to expose th Kafka broker, follow the instructions in the Access Kafka outside the cluster page.

For any other microservice, follow these steps to enable the LoadBalancer service:

  1. Open UMHLens / OpenLens and navigate to Network > Services.

  2. Select the service and click the Edit button.

  3. Scroll down to the status.loadBalancer section and change it to the following:

    status:
  loadBalancer:
    ingress:
    - ip: <external-ip>

    Replace <external-ip> with the external IP address of the node.

  4. Scroll to the spec.type section and change the value from ClusterIP to LoadBalancer.

  5. Click Save to apply the changes.

If you installed the United Manufacturing Hub on your local machine, either using the Management Console or the command line, you also need to map the port exposed by the k3d cluster to a port on your local machine. To do that, run the following command:

k3d cluster edit united-manufacturing-hub --port-add "<local-port>:<cluster-port>@server:0"

Replace <local-port> with a free port number on your local machine, and <cluster-port> with the port number of the service.

Port forwarding in UMHLens / OpenLens

If you don’t want to create a LoadBalancer service, effectively exposing the microservice to anyone that has access to the host IP address, you can use UMHLens / OpenLens to forward the port to your local machine.

  1. Open UMHLens / OpenLens and navigate to Network > Services.
  2. Select the service that you want to access.
  3. Scroll down to the Connection section and click the Forward… button.
  4. From the dialog, you can choose a port on your local machine to forward the cluster port from, or you can leave it empty to use a random port.
  5. Click Forward to apply the changes.
  6. If you left the checkbox Open in browser checked, then the service will open in your default browser.

You can see and manage the forwarded ports of your cluster in the Network > Port Forwarding section.

Port forwarding can be unstable, especially if the connection to the cluster is slow. If you are experiencing issues, try to create a LoadBalancer service instead.

Security considerations

MQTT broker

There are some security considerations to keep in mind when exposing the MQTT broker.

By default, the MQTT broker is configured to allow anonymous connections. This means that anyone can connect to the broker without providing any credentials. This is not recommended for production environments.

To secure the MQTT broker, you can configure it to require authentication. For that, you can either enable RBAC or set up HiveMQ PKI (recommended for production environments).

If you are using a version of the United Manufacturing Hub older than 0.9.10, then you need to change the ACL configuration to allow your MQTT client to connect to the broker.

Troubleshooting

LoadBalancer service stuck in Pending state

If the LoadBalancer service is stuck in the Pending state, it probably means that the host port is already in use. To fix this, edit the service and change the section spec.ports.port to a different port number.

What’s next

5.3.4 - Expose Grafana to the Internet

This page describes how to expose Grafana to the Internet.

This page describes how to expose Grafana to the Internet so that you can access it from outside the Kubernetes cluster.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Enable the ingress

To expose Grafana to the Internet, you need to enable the ingress.

  1. Open UMHLens / OpenLens and go to the Helm > Releases page.
  2. Click the Upgrade button and search for Grafana.
  3. Scroll down to the ingress section
  4. Set the enabled field to true.
  5. Add you domain name to the hosts field.
  6. Click Upgrade to apply the changes.

Remember to add a DNS record for your domain name that points to the external IP address of the Kubernetes host. You can find the external IP address of the Kubernetes host on the Nodes page in UMHLens / OpenLens.

What’s next

5.3.5 - Install Custom Drivers in NodeRed

This page describes how to install custom drivers in NodeRed.

NodeRed is running on Alpine Linux as non-root user. This means that you can’t install packages with apk. This tutorial shows you how to install packages with proper security measures.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Change the security context

  1. From the StatefulSets section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered to open the details page.

  2. Click the Edit button to open the StatefulSet’s configuration.

    Lens StatefulSet Edit
    Lens StatefulSet Edit

  3. Press Ctrl+F and search for securityContext.

  4. Set the values of the runAsUser field to 0, of fsGroup to 0, and of runAsNonRoot to false.

    ...
       securityContext:
         runAsUser: 0
   restartPolicy: Always
   terminationGracePeriodSeconds: 30
   dnsPolicy: ClusterFirst
   securityContext:
     runAsUser: 0
     runAsNonRoot: false
     fsGroup: 0
...

  5. Click Save.

Install the packages

  1. From the Pods section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Install the packages with apk:

    apk add <package>

    For example, to install unixodbc:

    apk add unixodbc

    You can find the list of available packages here.

  4. Exit the shell.

Revert the security context

For security reasons, you should revert the security context after you install the packages.

  1. From the StatefulSets section in UMHLens / OpenLens, click on united-manufacturing-hub-nodered to open the details page.

  2. Click the Edit button to open the StatefulSet’s configuration.

    Lens StatefulSet Edit
    Lens StatefulSet Edit

  3. Set the values of the runAsUser field to 1000, of fsGroup to 1000, and of runAsNonRoot to true.

    ...
       securityContext:
         runAsUser: 1000
   restartPolicy: Always
   terminationGracePeriodSeconds: 30
   dnsPolicy: ClusterFirst
   securityContext:
     runAsUser: 1000
     runAsNonRoot: true
     fsGroup: 1000
...

  4. Click Save.

What’s next

5.3.6 - Execute Kafka Shell Scripts

This page describes how to execute Kafka shell scripts.

When working with Kafka, you may need to execute shell scripts to perform administrative tasks. This page describes how to execute Kafka shell scripts.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open a shell in the Kafka container

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-kafka-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Navigate to the Kafka bin directory:

    cd /opt/bitnami/kafka/bin

  4. Execute any Kafka shell scripts. For example, to list all topics:

    ./kafka-topics.sh --list --zookeeper zookeeper:2181

  5. To exit the shell:

    exit

What’s next

5.3.7 - Reduce database size

This page describes how to reduce the size of the United Manufacturing Hub database.

Over time, time-series data can consume a large amount of disk space. To reduce the amount of disk space used by time-series data, there are three options:

  • Enable data compression. This reduces the required disk space by applying mathematical compression to the data. This compression is lossless, so the data is not changed in any way. However, it will take more time to compress and decompress the data. For more information, see how TimescaleDB compression works.
  • Enable data retention. This deletes old data that is no longer needed, by setting policies that automatically delete data older than a specified time. This can be beneficial for managing the size of the database, as well as adhering to data retention regulations. However, by definition, data loss will occur. For more information, see how TimescaleDB data retention works.
  • Downsampling. This is a method of reducing the amount of data stored by aggregating data points over a period of time. For example, you can aggregate data points over a 30-minute period, instead of storing each data point. If exact data is not required, downsampling can be useful to reduce database size. However, data may be less accurate.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open the database shell

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Enable data compression

To enable data compression, you need to execute the following SQL command from the database shell:

SELECT add_retention_policy('processvaluetable', INTERVAL '7 days');

This command will set a retention policy on the processvaluetable table, which will delete data older than 7 days.

Enable data retention

To enable data retention, you need to execute the following SQL command from the database shell:

SELECT add_compression_policy('processvaluetable', INTERVAL '7 days');

This command will set a compression policy on the processvaluetable table, which will compress data older than 7 days.

What’s next

5.3.8 - Delete Assets from the Database

This task shows you how to delete assets from the database.

This is useful if you have created assets by mistake, or to delete the ones that are no longer needed.

This task deletes data from the database. Make sure you have a backup of the database before you proceed.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Also, make sure to backup the database before you proceed. For more information, see Backing Up and Restoring the Database.

Open the database shell

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Choose the assets to delete

You have multiple choices to delete assets, like deleting a single asset, or deleting all assets in a location, or deleting all assets with a specific name.

To do so, you can customize the SQL command using different filters. Specifically, a combination of the following filters:

  • assetname
  • location
  • customer

To filter an SQL command, you can use the WHERE clause. For example, using all of the filters:

WHERE assetname = <my-asset> AND location = <my-location> AND customer = <my-customer>;

You can use any combination of the filters, even just one of them.

Here are some examples:

  • Delete all assets with the same name from any location and any customer:

    WHERE assetname = '<asset-name>'

  • Delete all assets in a specific location:

     WHERE location = '<location-name>'

  • Delete all assets with the same name in a specific location:

    WHERE assetname = '<asset-name>' AND location = '<location-name>'

  • Delete all assets with the same name in a specific location for a single customer:

    WHERE assetname = 'my-asset' AND location = 'my-location' AND customer = 'customer'

Delete the assets

Once you know the filters you want to use, you can use the following SQL commands to delete assets:

BEGIN;

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM shifttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM counttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM ordertable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM processvaluestringtable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM processvaluetable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM producttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM shifttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM statetable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM assettable WHERE id IN (SELECT id FROM assets_to_be_deleted);

COMMIT;

Optionally, you can add the following code before the last WITH statement if you used the track&trace feature:

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM producttagtable WHERE product_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM producttagstringtable WHERE product_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>), uniqueproducts_to_be_deleted AS (SELECT uniqueproductid FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted))
DELETE FROM productinheritancetable WHERE parent_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted) OR child_uid IN (SELECT uniqueproductid FROM uniqueproducts_to_be_deleted);

WITH assets_to_be_deleted AS (SELECT id FROM assettable <filter>)
DELETE FROM uniqueproducttable WHERE asset_id IN (SELECT id FROM assets_to_be_deleted);

What’s next

5.3.9 - Change the Language in Factoryinsight

This page describes how to change the language in Factoryinsight, in order to display the returned text in a different language.

You can change the language in Factoryinsight if you want to localize the returned text, like stop codes, to a different language.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Access the database shell

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Change the language

Execute the following command to change the language:

INSERT INTO configurationtable (customer, languagecode) VALUES ('factoryinsight', <code>) ON CONFLICT(customer) DO UPDATE SET languagecode=<code>;

where <code> is the language code. For example, to change the language to German, use 0.

Supported languages

Factoryinsight supports the following languages:

Supported languages
LanguageCode
German0
English1
Turkish2

What’s next

5.3.10 - Explore Cached Data

This page shows how to explore cached data in the United Manufacturing Hub.

When working with the United Manufacturing Hub, you might want to visualize information about the cached data. This page shows how you can access the cache and explore the data.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Open a shell in the cache Pod

  1. Open UMHLens / OpenLens and navigate to the Config > Secrets page.

  2. Get the cache password from the Secret redis-secret.

  3. From the Pods section click on united-manufacturing-hub-redis-master-0 to open the details page.

    If you have multiple cache Pods, you can select any of them.

  4. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  5. Enter the shell:

    redis-cli -a <cache-password>

  6. Now you can execute any command. For example, to get the number of keys in the cache, run:

    KEYS *

    Or, to get the cache size, run:

    DBSIZE

For more information about Redis commands, see the Redis documentation.

What’s next

5.3.11 - Optimize Time Consuming Queries

This page shows how to optimize the database in order to reduce the time needed to execute queries.

When you have a large database, it is possible that some queries take a long time to execute. This especially shows when you are using Grafana and the dropdown menu in the datasource takes a long time to load or does not load at all.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

Your United Manufacturing Hub must be at or later than version 0.9.4. To check the United Manufacturing Hub version, open UMHLens / OpenLens and go to Helm > Releases. The version is listed in the Version column.

Open a shell in the database container

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Create an index

Indexes are used to speed up queries. Run this query to create an index on the processvaluetable table:

CREATE INDEX ON processvaluetable(valuename, asset_id) WITH (timescaledb.transaction_per_chunk);

Rollback factoryinsight

If you have already created an index, you can rollback the factoryinsight deployment to version 0.9.4. This way it will use a less optimized but faster query, significantly reducing the execution time.

  1. From the Deployments section in UMHLens / OpenLens, click on united-manufacturing-hub-factoryinsight-deployment to open the details page.

  2. Click the Edit button to open the deployment’s configuration.

    Lens deployment Edit
    Lens deployment Edit

  3. Scroll down to the spec.containers section and change the image value to unitedmanufacturinghub/factoryinsight:0.9.4.
  4. Click Save.

What’s next

5.3.12 - Optimize Database Datatypes

This page describes how to change the datatype of some columns in the database in order to optimize the performance.

In version 0.9.5 and prior, some tables in the database were created with the varchar data type. This data type is not optimal for storing large amounts of data. In version 0.9.6, the data type of some columns was changed from varchar to text. This migration optimizes the database, by changing the data type of some columns from varchar to text.

Before you begin

You need to have a UMH cluster. If you do not already have a cluster, you can create one by using the Management Console.

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Alter the tables

Execute the following SQL statements:

ALTER TABLE assettable ALTER COLUMN assetid TYPE text;
ALTER TABLE assettable ALTER COLUMN location TYPE text;
ALTER TABLE assettable ALTER COLUMN customer TYPE text;
ALTER TABLE producttable ALTER COLUMN product_name TYPE text;
ALTER TABLE ordertable ALTER COLUMN order_name TYPE text;
ALTER TABLE configurationtable ALTER COLUMN customer TYPE text;
ALTER TABLE componenttable ALTER COLUMN componentname TYPE text;

Then confirm the changes by using the following SQL statements:

SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'assettable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'producttable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'ordertable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'configurationtable';
SELECT COLUMN_NAME, DATA_TYPE FROM information_schema.columns WHERE TABLE_NAME = 'componenttable';

5.4 - Backup & Recovery

This section contains information about how to backup and recover various components of the United Manufacturing Hub.

5.4.1 - Backup and Restore the United Manufacturing Hub

This page describes how to backup and restore the entire United Manufacturing Hub.

This page describes how to back up the following:

  • All Node-RED flows
  • All Grafana dashboards
  • The Helm values used for installing the united-manufacturing-hub release
  • All the contents of the United Manufacturing Hub database

It does not back up:

  • Additional databases other than the United Manufacturing Hub default database
  • TimescaleDB continuous aggregates: Follow the official documentation to learn how.
  • TimescaleDB policies: Follow the official documentation to learn how.
  • Everything else not included in the previous list

This procedure only works on Windows.

Before you begin

Download the backup scripts and extract the content in a folder of your choice.

For this task, you need to have PostgreSQL installed on your machine.

You also need to have enough space on your machine to store the backup. To check the size of the database, follow the steps below:

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

  1. Run the following command to get the size of the database:

    SELECT pg_size_pretty(pg_database_size('factoryinsight'));

Backup

Generate Grafana API Key

Create a Grafana API Token for an admin user by following these steps:

  1. Open the Grafana UI in your browser and log in with an admin user.
  2. Click on the Settings icon in the left sidebar and select API Keys.
  3. Give the API key a name and change its role to Admin.
  4. Optionally set an expiration date.
  5. Click Add.
  6. Copy the generated API key and save it for later.

Stop workloads

To prevent data inconsistencies, you need to temporarily stop the MQTT and Kafka Brokers.

  1. In UMHLens / OpenLens go to the Workloads > StatefulSets tab.
  2. Select the united-manufacturing-hub-kafka StatefulSet

  3. Click the Scale button to select the number of replicas for the StatefulSet.

    Lens StatefulSet Scale
    Lens StatefulSet Scale

  4. Set the number of replicas to 0 and click Scale.
  5. Repeat the process for the united-manufacturing-hub-hivemqce StatefulSet.

Backup using the script

The backup script is located inside the folder you downloaded earlier.

  1. Open a terminal and navigate inside the folder.

    cd <FOLDER_PATH>

  2. Run the script:

    .\backup.ps1 -IP <IP_OF_THE_SERVER> -GrafanaToken <GRAFANA_API_KEY> -KubeconfigPath <PATH_TO_KUBECONFIG>

    You can find a list of all available parameters down below.

    If OutputPath is not set, the backup will be stored in the current folder.

This script might take a while to finish, depending on the size of your database and your connection speed.

If the connection is interrupted, there is currently no option to resume the process, therefore you will need to start again.

Here is a list of all available parameters:

Available parameters
ParameterDescriptionRequiredDefault value
GrafanaTokenGrafana API keyYes
IPIP of the cluster to backupYes
KubeconfigPathPath to the kubeconfig fileYes
DatabaseDatabaseName of the databse to backupNofactoryinsight
DatabasePasswordPassword of the database userNochangeme
DatabasePortPort of the databaseNo5432
DatabaseUserDatabase userNofactoryinsight
DaysPerJobNumber of days worth of data to backup in each parallel jobNo31
EnableGpgEncryptionSet to true if you want to encrypt the backupNofalse
EnableGpgSigningSet to true if you want to sign the backupNofalse
GpgEncryptionKeyIdID of the GPG key used for encryptionNo
GpgSigningKeyIdID of the GPG key used for signingNo
GrafanaPortExternal port of the Grafana serviceNo8080
OutputPathPath to the folder where the backup will be storedNoCurrent folder
ParallelJobsNumber of parallel job backups to runNo4
SkipDiskSpaceCheckSkip checking available disk spaceNofalse
SkipGpgQuestionsSet to true if you want to sign or encrypt the backupNofalse

Restore

Each component of the United Manufacturing Hub can be restored separately, in order to allow for more flexibility and to reduce the damage in case of a failure.

Cluster configuration

To restore the Kubernetes cluster, execute the .\restore-helm.ps1 script with the following parameters:

.\restore-helm.ps1 -KubeconfigPath <PATH_TO_KUBECONFIG> -BackupPath <PATH_TO_BACKUP_FOLDER>

Verify that the cluster is up and running by opening UMHLens / OpenLens and checking if the workloads are running.

Grafana dashboards

To restore the Grafana dashboards, you first need to create a Grafana API Key for an admin user in the new cluster by following these steps:

  1. Open the Grafana UI in your browser and log in with an admin user.
  2. Click on the Settings icon in the left sidebar and select API Keys.
  3. Give the API key a name and change its role to Admin.
  4. Optionally set an expiration date.
  5. Click Add.
  6. Copy the generated API key and save it for later.

Then, on your local machine, execute the .\restore-grafana.ps1 script with the following parameters:

.\restore-grafana.ps1 -FullUrl http://<IP_OF_THE_SERVER>:8080 -Token <GRAFANA_API_KEY> -BackupPath <PATH_TO_BACKUP_FOLDER>

Restore Node-RED flows

To restore the Node-RED flows, execute the .\restore-nodered.ps1 script with the following parameters:

.\restore-nodered.ps1 -KubeconfigPath <PATH_TO_KUBECONFIG> -BackupPath <PATH_TO_BACKUP_FOLDER>

Restore the database

To restore the database, execute the .\restore-timescale.ps1 script with the following parameters:

.\restore-timescale.ps1 -Ip <IP_OF_THE_SERVER> -BackupPath <PATH_TO_BACKUP_FOLDER> -PatroniSuperUserPassword <DATABASE_PASSWORD>

What’s next

5.4.2 - Backup and Restore Database

This page describes how to backup and restore the database.

Before you begin

For this task, you need to have PostgreSQL installed on your machine.

You also need to have enough space on your machine to store the backup. To check the size of the database, follow the steps below:

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

  1. Run the following command to get the size of the database:

    SELECT pg_size_pretty(pg_database_size('factoryinsight'));

Backing up the database

Follow these steps to create a backup of the factoryinsight database on your machine:

  1. Open a terminal, and using the cd command, navigate to the folder where you want to store the backup. For example:

    
   cd C:\Users\user\backups
   

    
   cd /Users/user/backups
   

    
   cd /home/user/backups

    If the folder does not exist, you can create it using the mkdir command or your file manager.

  2. Run the following command:

    pg_dump -h <REMOTE_HOST> -p 5432 -U factoryinsight -Fc -f <BACKUP_NAME>.bak factoryinsight
    • <REMOTE_HOST> is the IP of the server where the database is running. Use localhost if you installed the United Manufacturing Hub using k3d.
    • <BACKUP_NAME> is the name of the backup file.

Grafana database

If you want to backup the Grafana database, you can follow the same steps as above, but you need to replace any occurence of factoryinsight with grafana.

Additionally, you also need to write down the credentials in the grafana-secret Secret, as they will be needed to access the dashboard after restoring the database.

Restoring the database

This section is untested. Please report any issues you encounter.

For this section, we assume that you are restoring the data to a fresh United Manufacturing Hub installation with an empty database.

Copy the backup file to the database pod

  1. Open UMHLens / OpenLens.

  2. Launch a new terminal sesstion by clicking on the + button in the bottom-left corner of the window.

  3. Run the following command to copy the backup file to the database pod:

    kubectl cp /path/to/local/backup.bak united-manufacturing-hub-timescaledb-0:/tmp/backup.bak

    Replace /path/to/local/backup.bak with the path to the backup file on your machine.

This step could take a while depending on the size of the backup file.

Temporarly disable kafkatopostrgesql

  1. Navigate to Workloads > Deployments.
  2. Select the united-manufacturing-hub-kafkatopostgresql Deployment.

  3. Click the Scale button to select the number of replicas for the Deployment.

    Lens Deployment Scale
    Lens Deployment Scale

  4. Scale the number of replicas to 0.

Open a shell in the database pod

  1. From the Pod section in UMHLens / OpenLens, click on united-manufacturing-hub-timescaledb-0 to open the details page.

  2. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  3. Enter the postgres shell:

    psql

  4. Connect to the database:

    \c factoryinsight

Restore the database

  1. Drop the existing database:

    DROP DATABASE factoryinsight;

  2. Create a new database:

    CREATE DATABASE factoryinsight;
\c factoryinsight
CREATE EXTENSION IF NOT EXISTS timescaledb;

  3. Put the database in maintenance mode:

    SELECT timescaledb_pre_restore();

  4. Restore the database:

    \! pg_restore -Fc -d factoryinsight /tmp/backup.bak

  5. Take the database out of maintenance mode:

    SELECT timescaledb_post_restore();

Enable kafkatopostgresql

  1. Navigate to Workloads > Deployments.
  2. Select the united-manufacturing-hub-kafkatopostgresql Deployment.

  3. Click the Scale button to select the number of replicas for the Deployment.

    Lens Deployment Scale
    Lens Deployment Scale

  4. Scale the number of replicas to the original value, usually 1.

What’s next

5.4.3 - Import and Export Node-RED Flows

This page describes how to import and export Node-RED flows.

Export Node-RED Flows

To export Node-RED flows, please follow the steps below:

  1. Access Node-RED by navigating to http://<CLUSTER-IP>:1880/nodered in your browser. Replace <CLUSTER-IP> with the IP address of your cluster, or localhost if you are running the cluster locally.

  2. From the top-right menu, select Export.

  3. From the Export dialog, select wich nodes or flows you want to export.

  4. Click Download to download the exported flows, or Copy to clipboard to copy the exported flows to the clipboard.

    ExportWindow
    ExportWindow

The credentials of the connector nodes are not exported. You will need to re-enter them after importing the flows.

Import Node-RED Flows

To import Node-RED flows, please follow the steps below:

  1. Access Node-RED by navigating to http://<CLUSTER-IP>:1880/nodered in your browser. Replace <CLUSTER-IP> with the IP address of your cluster, or localhost if you are running the cluster locally.

  2. From the top-right menu, select Import.

  3. From the Import dialog, select the file containing the exported flows, or paste the exported flows from the clipboard.

  4. Click Import to import the flows.

    ImportWindow
    ImportWindow

5.5 - Security

This section contains information about how to secure the United Manufacturing Hub.

5.5.1 - Change VerneMQ ACL Configuration

This page describes how to change the ACL configuration to allow more users to publish to the MQTT broker

Change VerneMQ ACL configuration

  1. Open UMHLens / OpenLens

  2. Navigate to Helm > Releases.

  3. Select the united-manufacturing-hub release and click Upgrade.

  4. Find the _000_commonConfig.infrastrucutre.mqtt section.

  5. Update the AclConfig value to allow unrestricted access, for example:

    AclConfig: |
  pattern # allow all

  6. Click Upgrade to apply the changes.

What’s next

5.5.2 - Enable RBAC for the MQTT Broker

This page describes how to enable Role-Based Access Control (RBAC) for the MQTT broker.

Enable RBAC

  1. Open UMHLens / OpenLens
  2. Navigate to Helm > Releases.
  3. Select the united-manufacturing-hub release and click Upgrade.
  4. Find the mqtt_broker section.
  5. Locate the rbacEnabled parameter and change its value from false to true.
  6. Click Upgrade.

Now all MQTT connections require password authentication with the following defaults:

  • Username: node-red
  • Password: INSECURE_INSECURE_INSECURE

Change default credentials

  1. Open UMHLens / OpenLens

  2. Navigate to Workloads > Pods.

  3. Select the united-manufacturing-hub-hivemqce-0 Pod.

  4. Click the Pod Shell button to open a shell in the container.

    Lens Pod Shell
    Lens Pod Shell

  5. Navigate to the installation directory of the RBAC extension.

    cd extensions/hivemq-file-rbac-extension/

  6. Generate a password hash with this command.

    java -jar hivemq-file-rbac-extension-<version>.jar -p <password>
    • Replace <version> with the version of the HiveMQ CE extension. If you are not sure which version is installed, you can press Tab after typing java -jar hivemq-file-rbac-extension- to autocomplete the version.
    • Replace <password> with your desired password. Do not use any whitespaces.

  7. Copy the output of the command. It should look similar to this:

    $2a$10$Q8ZQ8ZQ8ZQ8ZQ8ZQ8ZQ8Zu

  8. Navigate to Config > ConfigMaps.

  9. Select the united-manufacturing-hub-hivemqce-extension ConfigMap.

  10. Click the Edit button to open the ConfigMap editor.

  11. In the data.credentials.xml section, replace the strings inbetween the <password> tags with the password hash generated in step 7.

    You can use a different password for each different microservice. Just remember that you will need to update the configuration in each one to use the new password.

  12. Click Save to apply the changes.

  13. Go back to Workloads > Pods and select the united-manufacturing-hub-hivemqce-0 Pod.

  14. Click the Delete button to delete the Pod.

    Lens Pod Delete
    Lens Pod Delete

What’s next

5.5.3 - Setup PKI for the MQTT Broker

This page describes how to setup the Public Key Infrastructure (PKI) for the MQTT broker.

If you want to use MQTT over TLS (MQTTS) or Secure Web Socket (WSS) you need to setup a Public Key Infrastructure (PKI).

Read the blog article about secure communication in IoT to learn more about encryption and certificates.

Structure overview

The Public Key Infrastructure for HiveMQ consists of two Java Key Stores (JKS):

  • Keystore: The Keystore contains the HiveMQ certificate and private keys. This store must be confidential, since anyone with access to it could generate valid client certificates and read or send messages in your MQTT infrastructure.
  • Truststore: The Truststore contains all the clients public certificates. HiveMQ uses it to verify the authenticity of the connections.

Before you begin

You need to have the following tools installed:

  • OpenSSL. If you are using Windows, you can install it with Chocolatey.
  • Java

Create a Keystore

Open a terminal and run the following command:

keytool -genkey -keyalg RSA -alias hivemq -keystore hivemq.jks -storepass <password> -validity <days> -keysize 4096 -dname "CN=united-manufacturing-hub-mqtt" -ext "SAN=IP:127.0.0.1"

Replace the following placeholders:

  • <password>: The password for the keystore. You can use any password you want.
  • <days>: The number of days the certificate should be valid.

The command runs for a few minutes and generates a file named hivemq.jks in the current directory, which contains the HiveMQ certificate and private key.

If you want to explore the contents of the keystore, you can use Keystore Explorer.

Generate client certificates

Open a terminal and create a directory for the client certificates:

mkdir pki

Follow these steps for each client you want to generate a certificate for.

  1. Create a new key pair:

    openssl req -new -x509 -newkey rsa:4096 -keyout "pki/<servicename>-key.pem" -out "pki/<servicename>-cert.pem" -nodes -days <days> -subj "/CN=<servicename>"

  2. Convert the certificate to the correct format:

    openssl x509 -outform der -in "pki/<servicename>-cert.pem" -out "pki/<servicename>.crt"

  3. Import the certificate into the Truststore:

    keytool -import -file "pki/<servicename>.crt" -alias "<servicename>" -keystore hivemq-trust-store.jks -storepass <password>

Replace the following placeholders:

  • <servicename> with the name of the client. Use the service name from the Network > Services tab in UMHLens / OpenLens.
  • <days> with the number of days the certificate should be valid.
  • <password> with the password for the Truststore. You can use any password you want.

Import the PKI into the United Manufacturing Hub

First you need to encode in base64 the Keystore, the Truststore and all the PEM files. Use the following script to encode everything automatically:

Get-ChildItem .\ -Recurse -Include *.jks,*.pem | ForEach-Object {
    $FileContent = Get-Content $_ -Raw
    $fileContentInBytes = [System.Text.Encoding]::UTF8.GetBytes($FileContent)
    $fileContentEncoded = [System.Convert]::ToBase64String($fileContentInBytes)
    $fileContentEncoded > $_".b64"
    Write-Host $_".b64 File Encoded Successfully!"
}


find ./ -regex '.*\.jks\|.*\.pem' -exec openssl base64 -A -in {} -out {}.b64 \;

You could also do it manually with the following command:

openssl base64 -A -in <filename> -out <filename>.b64

Now you can import the PKI into the United Manufacturing Hub. To do so:

  1. Open UMHLens / OpenLens.
  2. Navigate to Helm > Releases.
  3. Select the united-manufacturing-hub release.
  4. Click the Upgrade button.
  5. Find the _000_commonConfig.infrastructure.mqtt.tls section.
  6. Update the value of the keystoreBase64 field with the content of the hivemq.jks.b64 file and the value of the keystorePassword field with the password you used for the keystore.
  7. Update the value of the truststoreBase64 field with the content of the hivemq-trust-store.jks.b64 file and the value of the truststorePassword field with the password you used for the truststore.
  8. Update the value of the <servicename>.cert field with the content of the <servicename>-cert.pem.b64 file and the value of the <servicename>.key field with the content of the <servicename>-key.pem.b64 file.
  9. Click the Upgrade button to apply the changes.

What’s next

6 - What's new

This section contains information about the new features and changes in the United Manufacturing Hub.

For release highlights, deprecations, and breaking changes in the United Manufacturing Hub, refer to these “What’s new” pages for each version.

For a complete list of every change, with links to pull requests and related issues, refer to the release notes.

United Manufacturing Hub 0.9

6.1 - What's New in Version 0.9.15

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.15.

Welcome to United Manufacturing Hub version 0.9.15! In this release we added support for the UNS data model, by introducing a new microservice, Data Bridge.

For a complete list of changes, refer to the release notes.

Data Bridge

Data-bridge is a microservice specifically tailored to adhere to the UNS data model. It consumes topics from a message broker, translates them to the proper format and publishes them to the other message broker.

It can consume from and publish to both Kafka and MQTT brokers, whether they are local or remote.

It’s main purpose is to merge messages from multiple topics into a single topic, using the message key to identify the source topic.

Updated dependencies

We updated the following dependencies:

  • RedPanda to version 23.2.8
  • HiveMQ to the community edition version 2023.7

6.2 - What's New in Version 0.9.14

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.14.

Welcome to United Manufacturing Hub version 0.9.14! In this release we changed the Kafka broker from Apache Kafka to RedPanda, which is a Kafka-compatible event streaming platform. We also started migrating to a different kafka library in our micoservices, which will allow full ARM support in the future. Finally, we tweaked the overall resource usage of the United Manufacturing Hub to improve performance and efficiency, along with some bug fixes.

For a complete list of changes, refer to the release notes.

RedPanda

RedPanda is a Kafka-compatible event streaming platform. It is built with modern hardware in mind and utilizes multi-core CPUs efficiently, which can result in better performance compared to Kafka. RedPanda also offers lower latency and higher throughput, making it a better fit for real-time use cases in IIoT applications. Additionally, RedPanda has a simpler setup and management process compared to Kafka, which can save time and resources for development teams. Finally, RedPanda is fully compatible with Kafka’s API, allowing for a seamless transition for existing Kafka users.

Overall, Redpanda can provide improved performance and efficiency for IIoT applications that require real-time data processing and management with a lower setup and management cost.

Sarama Kafka Library

We started migrating our microservices to use the Sarama Kafka library. This library is written in Go and is fully compatible with RedPanda. This change will allow us to support ARM-based devices in the future, which will be useful for edge computing use cases. Addedd bonus is that Sarama is faster and requires less memory than the previous library.

For now we only migrated the following microservices:

  • barcodereader
  • kafka-init (used as an init container for components that communicate with Kafka)
  • mqtt-kafka-bridge

Resources tweaking

With this release we tweaked the resource requests of each default component of the United Manufacturing Hub to respect the minimum requirements of 4 cores and 8GB of RAM. This allowed us to increase the memory allocated for the MQTT broker, resulting in solving the common Out Of Memory issue that caused the broker to restart.

Be sure to follow the upgrade guide to adjust your resources accordingly.

The following table shows the new resource requests and limits when deploying the United Manufacturing Hub with the default configuration or with all the components enabled. CPU values are expressed in millicores and memory values are expressed in mebibytes.

resources
ResourceRequestsLimits
CPU (default values)1080m (27%)1890m (47%)
Memory (default values)1650Mi (21%)2770Mi (35%)
CPU (all components)2002m (50%)2730m (68%)
Memory (all components)2873Mi (36%)3578Mi (45%)

The requested resources are the ones immediately allocated to the container when it starts, and the limits are the maximum amount of resources that the container can (but is not forced to) use. For more information about Kubernetes resources, refer to the official documentation.

Container registry

We moved our container registry from Docker Hub to GitHub Container Registry. This change won’t affect the way you deploy the United Manufacturing Hub, but it will allow us to better manage our container images and provide a better experience for our developers. For the time being, we will continue to publish our images to Docker Hub, but we will eventually deprecate the old registry.

Others

  • Implemented a new test build to detect race conditions in the codebase. This will help us to improve the stability of the United Manufacturing Hub.
  • All our custom images now run as non-root by default, except for the ones that require root privileges.
  • The custom microservices now allow to change the type of Service used to expose them by setting serviceType field.
  • Added an SQL trigger function that deletes duplicate records from the statetable table after insertion.
  • Enhanced the environment variables validation in the codebase.
  • Added possibility to set the aggregation interval when calculating the throughput of an asset.
  • Various dependencies has been updated to their latest version.

6.3 - What's New in Version 0.9.13

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.13.

Welcome to United Manufacturing Hub version 0.9.13! This is a minor release that only updates the new metrics feature.

For a complete list of changes, refer to the release notes.

6.4 - What's New in Version 0.9.12

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.12.

Welcome to United Manufacturing Hub version 0.9.12! Read on to learn about the new features of the UMH Datasource V2 plugin for Grafana, Redis running in standalone mode, and more.

For a complete list of changes, refer to the release notes.

Grafana

New Grafana version

Grafana has been upgraded to version 9.4.3. This introduces new search and navigation features, a redesigned details section of the logs, and a new data source connection page.

Head over to the Grafana release notes to learn more about the new features.

New Node-RED version

We have upgraded Node-RED to version 3.0.2. Checkout the Node-RED release notes for more information.

UMH Datasource V2 plugin

The latest update to the datasource has incorporated typesafe JSON parsing, significantly enhancing the overall performance and dependability of the plugin. This implementation ensures that the parsing process strictly adheres to predefined data types, eliminating the possibility of unexpected errors or data corruption that can occur with loosely-typed JSON parsing.

Redis in standalone mode

Redis, the service used for caching, is now deployed in standalone mode. This change introduces these benefits:

  • Simplicity: Running Redis in standalone mode is simpler than using a master-replica topology with Sentinel. With standalone mode, there is only one Redis instance to manage, whereas with master-replica, you need to manage multiple Redis instances and the Sentinel process. This simplicity can reduce complexity and make it easier to manage Redis instances.
  • Lower Overhead: Standalone mode has lower overhead than using a master-replica topology with Sentinel. In a master-replica topology, there is a communication overhead between the master and the replicas, and Sentinel adds additional overhead for monitoring and failover management. In contrast, standalone mode does not have this overhead.
  • Better Performance: Since standalone mode does not have the overhead of master-replica topology with Sentinel, it can provide better performance. Standalone mode provides faster response times and can handle more requests per second than a master-replica topology with Sentinel.

That being said, it’s important to note that a master-replica topology with Sentinel provides higher availability and failover capabilities than standalone mode.

All basic services are now exposed by a LoadBalancer Service

The MQTT Broker, Kafka Broker, and Kafka Console are now exposed by a LoadBalancer Service, along with the Database, Grafana and Node-RED. This change makes it easier to access these services from outside the cluster, as they are now accessible via the IP address of the cluster.

When installing the United Manufacturing Hub locally, the cluster ports are automatically mapped to the host ports. This means that you can access the services from your browser by using localhost and the port number.

Read more about connecting to the services from outside the cluster in the related documentation.

Metrics

We introduced an optional microservice that can be used to collect metrics about the system, like OS, CPU, memory, hostname and average load. These metrics are then sent to our server for analysis, and are completely anonymous. This microservice is enabled by default, but can be disabled by setting the _000_commonConfig.metrics.enabled value to false in the values.yaml file.

Click to see an example metric
{
   "OS":"linux",
   "Arch":"amd64",
   "Memory":{
      "total":16435666944,
      "available":11555106816,
      "used":4404510720,
      "usedPercent":26.798490958761544,
      "free":574394368,
      "active":3613691904,
      "inactive":10843209728,
      "wired":0,
      "laundry":0,
      "buffers":588361728,
      "cached":10868400128,
      "writeback":0,
      "dirty":122880,
      "writebacktmp":0,
      "shared":155168768,
      "slab":978030592,
      "sreclaimable":766824448,
      "sunreclaim":211206144,
      "pagetables":32157696,
      "swapcached":17887232,
      "commitlimit":12512800768,
      "committedas":16789483520,
      "hightotal":0,
      "highfree":0,
      "lowtotal":0,
      "lowfree":0,
      "swaptotal":4294967296,
      "swapfree":4165865472,
      "mapped":1214676992,
      "vmalloctotal":35184372087808,
      "vmallocused":60112896,
      "vmallocchunk":0,
      "hugepagestotal":0,
      "hugepagesfree":0,
      "hugepagesize":2097152
   },
   "CPUInfo":[
      {
         "cpu":0,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"0",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":1,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"0",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":2,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"1",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":3,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"1",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":4,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"2",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":5,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"2",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":6,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"3",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":7,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"3",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":8,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"4",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":9,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"4",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":10,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"5",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":11,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"5",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":12,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"6",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":13,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"6",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":14,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"7",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      },
      {
         "cpu":15,
         "vendorId":"AuthenticAMD",
         "family":"25",
         "model":"80",
         "stepping":0,
         "physicalId":"0",
         "coreId":"7",
         "cores":1,
         "modelName":"AMD Ryzen 9 5900HX with Radeon Graphics",
         "mhz":3293.73,
         "cacheSize":512,
         "flags":[
            "fpu",
            "vme",
            "de",
            "pse",
            "tsc",
            "msr",
            "pae",
            "mce",
            "cx8",
            "apic",
            "sep",
            "mtrr",
            "pge",
            "mca",
            "cmov",
            "pat",
            "pse36",
            "clflush",
            "mmx",
            "fxsr",
            "sse",
            "sse2",
            "ht",
            "syscall",
            "nx",
            "mmxext",
            "fxsr_opt",
            "pdpe1gb",
            "rdtscp",
            "lm",
            "constant_tsc",
            "rep_good",
            "nopl",
            "tsc_reliable",
            "nonstop_tsc",
            "cpuid",
            "extd_apicid",
            "pni",
            "pclmulqdq",
            "ssse3",
            "fma",
            "cx16",
            "sse4_1",
            "sse4_2",
            "movbe",
            "popcnt",
            "aes",
            "xsave",
            "avx",
            "f16c",
            "rdrand",
            "hypervisor",
            "lahf_lm",
            "cmp_legacy",
            "svm",
            "cr8_legacy",
            "abm",
            "sse4a",
            "misalignsse",
            "3dnowprefetch",
            "osvw",
            "topoext",
            "perfctr_core",
            "ssbd",
            "ibrs",
            "ibpb",
            "stibp",
            "vmmcall",
            "fsgsbase",
            "bmi1",
            "avx2",
            "smep",
            "bmi2",
            "erms",
            "invpcid",
            "rdseed",
            "adx",
            "smap",
            "clflushopt",
            "clwb",
            "sha_ni",
            "xsaveopt",
            "xsavec",
            "xgetbv1",
            "xsaves",
            "clzero",
            "xsaveerptr",
            "arat",
            "npt",
            "nrip_save",
            "tsc_scale",
            "vmcb_clean",
            "flushbyasid",
            "decodeassists",
            "pausefilter",
            "pfthreshold",
            "v_vmsave_vmload",
            "umip",
            "vaes",
            "vpclmulqdq",
            "rdpid",
            "fsrm"
         ],
         "microcode":"0xffffffff"
      }
   ],
   "Host":{
      "hostname":"7bd935ddb2b727e7ec31c2b17f238cd68a05eddae3de2f3f30df60128cf06e1d82b4643d6a1dfd54310fbc00d0f8e248a4ea2726b7e84bf1a420330d527253ee",
      "uptime":15610,
      "bootTime":1680249794,
      "procs":1,
      "os":"linux",
      "platform":"alpine",
      "platformFamily":"alpine",
      "platformVersion":"3.17.2",
      "kernelVersion":"5.15.90.1-microsoft-standard-WSL2",
      "kernelArch":"x86_64",
      "virtualizationSystem":"docker",
      "virtualizationRole":"guest",
      "hostid":"48f4b69b63d90af6691ee87361c0419af89c13d4504c4c85329599f5e0ea075e1668ed4788d6caaa2ffc299ca461b0d46e32e82c0c5405b23b29dcf8e5a8a1dc"
   },
   "Load":{
      "load1":0.05,
      "load5":0.19,
      "load15":0.49
   }
}

Sarama in MQTT-Kafka Bridge

We replaced confluent kafka with Sarama in the MQTT-Kafka Bridge. This increased the performance & stability and is a first step towards ARM compatibility.

Automated testing

We have added automated end-to-end testing to the United Manufacturing Hub. This includes testing the installation and the upgrading of the United Manufacturing Hub, as well as testing the functionality of the microservices.

Deprecations

Cameraconnect

The Cameraconnect microservice has been deprecated and removed from the United Manufacturing Hub. It’s development has been taken over by Anticipate.

Blob storage

The blob storage service has been deprecated and removed from the United Manufacturing Hub. This includes the MinIO Operator and Tenant, and the MQTT to Blob microservice.

Fixes

Many fixes have been made to the United Manufacturing Hub, including known issues for Sensorconnect and MQTT Bridge.

6.5 - What's New in Version 0.9.11

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.11.

Welcome to United Manufacturing Hub version 0.9.11! This patch introduces only minor bugfixes for Factoryinput and Sensorconnect.

For a complete list of changes, refer to the release notes.

6.6 - What's New in Version 0.9.10

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.10.

Welcome to United Manufacturing Hub version 0.9.10! In this release, we have changed the MQTT broker to HiveMQ and the Kafka console to RedPanda Console. A new OPC UA server simulator has been added, along with a new API service to connect to Factoryinsight from outside the cluster, especially tailored for usage with Tulip. Grafana now comes with presinstalled plugins and datasources, and the UMH datasource V2 supports grouping of custom tags.

For a complete list of changes, refer to the release notes.

MQTT Broker

The MQTT broker has been changed from VerneMQ to HiveMQ. This change won’t affect the end user, but it will allow us to better maintain the MQTT broker in the future.

Read our comparison of MQTT brokers to learn more about the differences between VerneMQ and HiveMQ.

Kafka Console

The Kowl project as been acquired by RedPanda and is now called RedPanda Console. The functionalities are mostly the same.

OPC UA Server Simulator

A new data simulator for OPC UA has been added. It is based on the OPC/UA simulator by Amine, and it allows you to simulate OPC UA servers in order to test the United Manufacturing Hub.

Grafana

Default plugins

Grafana now comes with the following plugins preinstalled:

  • ACE.SVG by Andrew Rodgers
  • Button Panel by UMH Systems Gmbh
  • Button Panel by CloudSpout LLC
  • Discrete by Natel Energy
  • Dynamic Text by Marcus Olsson
  • FlowCharting by agent
  • Pareto Chart by isaozler
  • Pie Chart (old) by Grafana Labs
  • Timepicker Buttons Panel by williamvenner
  • UMH Datasource by UMH Systems Gmbh
  • UMH Datasource V2 by UMH Systems Gmbh
  • Untimely by factry
  • Worldmap Panel by Grafana Labs

Grouping of custom tags

The UMH datasource V2 now supports grouping of custom tags. This allows you to group processValues by a common prefix, and then use the group name as a variable in Grafana.

Tulip connector

A new API service has been added to connect to Factoryinsight from outside the cluster. This service is especially tailored for usage with Tulip, and it allows you to connect to Factoryinsight from outside the cluster.

Read more about the Tulip connector.

6.7 - What's New in Version 0.9.9

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.9.

Welcome to United Manufacturing Hub version 0.9.9! This version introduces the PackML-MQTT-Simulator, to simulate a PackML state machine and publish the state changes to MQTT. It also includes new liveness probes for some of the Pods and minor fixes.

For a complete list of changes, refer to the release notes.

PackML-MQTT-Simulator

The PackML-MQTT-Simulator is is a virtual line that interfaces using PackML implemented over MQTT. It allows you to simulate a PackML state machine and publish the state changes to MQTT.

6.8 - What's New in Version 0.9.8

This section contains information about the new features and changes in the United Manufacturing Hub introduced in version 0.9.8.

Welcome to United Manufacturing Hub version 0.9.8! Read on to learn about the new Factoryinsight API V2 and the related datasource plugin for Grafana with support for historian functionalities.

For a complete list of changes, refer to the release notes.

Historian functionalities

The new v2 API of Factoryinsight now supports historian functionalities. This means that you can now query the history of your data and visualize it in Grafana. The new datasource plugin for Grafana supports the Time Bucket Aggregation, which allows you to aggregate your data by values like the average, minimum or maximum.

7 - Reference

This section of the United Manufacturing Hub documentation contains references.

7.1 - Helm Chart

This page describes the Helm Chart of the United Manufacturing Hub and the possible configuration options.

An Helm chart is a package manager for Kubernetes that simplifies the installation, configuration, and deployment of applications and services. It contains all the necessary Kubernetes manifests, configuration files, and dependencies required to run a particular application or service. One of the main advantages of Helm is that it allows to define the configuration of the installed resources in a single YAML file, called values.yaml. Helm provides great documentation on this process.

The Helm Chart of the United Manufacturing Hub is composed of both custom microservices and third-party applications. If you want a more in-depth view of the architecture of the United Manufacturing Hub, you can read the Architecture overview page.

Helm Chart structure

Custom microservices

The Helm Chart of the United Manufacturing Hub is composed of the following custom microservices:

  • barcodereader: reads the input from a barcode reader and sends it to the MQTT broker for further processing.
  • customMicroservice: a template for deploying any number of custom microservices.
  • data-bridge: transfers data between two Kafka or MQTT brokers, transforming the data following the UNS data model.
  • factoryinput: provides REST endpoints for MQTT messages.
  • factoryinsight: provides REST endpoints to fetch data and calculate KPIs.
  • grafanaproxy: provides a proxy to the backend services.
  • MQTT Simulator: simulates sensors and sends the data to the MQTT broker for further processing.
  • kafka-bridge: connects Kafka brokers on different Kubernetes clusters.
  • kafkatopostgresql: stores the data from the Kafka broker in a PostgreSQL database.
  • mqtt-kafka-bridge: connects the MQTT broker and the Kafka broker.
  • mqttbridge: connects MQTT brokers on different Kubernetes clusters.
  • opcuasimulator: simulates OPC UA servers and sends the data to the MQTT broker for further processing.
  • packmlmqttsimulator: simulates a PackML state machine and sends the data to the MQTT broker for further processing.
  • sensorconnect: connects to a sensor and sends the data to the MQTT and Kafka brokers for further processing.
  • tulip-connector: exposes internal APIs to the internet, especially tailored for the Tulip platform.

Third-party applications

The Helm Chart of the United Manufacturing Hub is composed of the following third-party applications:

  • Grafana: a visualization and analytics software.
  • HiveMQ: an MQTT broker.
  • Node-RED: a programming tool for wiring together hardware devices, APIs and online services.
  • Redis: an in-memory data structure store, used for cache.
  • RedPanda: a Kafka-compatible distributed event streaming platform.
  • RedPanda Console: a web-based user interface for RedPanda.
  • TimescaleDB: an open-source time-series SQL database.

Configuration options

The Helm Chart of the United Manufacturing Hub can be configured by setting values in the values.yaml file. This file has three main sections that can be used to configure the applications:

  • customers: contains the definition of the customers that will be created during the installation of the Helm Chart. This section is optional, and it’s used only by factoryinsight and factoryinput.
  • _000_commonConfig: contains the basic configuration options to customize the United Manufacturing Hub, and it’s divided into sections that group applications with similar scope, like the ones that compose the infrastructure or the ones responsible for data processing. This is the section that should be mostly used to configure the microservices.
  • _001_customMicroservices: used to define the configuration of custom microservices that are not included in the Helm Chart.

After those three sections, there are the specific sections for each microservice, which contain their advanced configuration. This is the so called Danger Zone, because the values in those sections should not be changed, unlsess you absolutely know what you are doing.

When a parameter contains . (dot) characters, it means that it is a nested parameter. For example, in the tls.factoryinput.cert parameter the cert parameter is nested inside the tls.factoryinput section, and the factoryinput section is nested inside the tls section.

Customers

The customers section contains the definition of the customers that will be created during the installation of the Helm Chart. It’s a simple dictionary where the key is the name of the customer, and the value is the password.

For example, the following snippet creates two customers:

customers:
  customer1: password1
  customer2: password2

Common configuration options

The _000_commonConfig contains the basic configuration options to customize the United Manufacturing Hub, and it’s divided into sections that group applications with similar scope.

The following table lists the configuration options that can be set in the _000_commonConfig section:

_000_commonConfig section parameters
ParameterDescriptionTypeAllowed valuesDefault
datainputThe configuration of the microservices used to input data.objectSee belowSee below
datamodel_v2The configuration related to the UNS data model.objectSee belowSee below
dataprocessingThe configuration of the microservices used to process data.objectSee belowSee below
datasourcesThe configuration of the microservices used to acquire data.objectSee belowSee below
datastorageThe configuration of the microservices used to store data.objectSee belowSee below
debugThe configuration for the debug mode.objectSee belowSee below
infrastructureThe configuration of the microservices used to provide infrastructure services.objectSee belowSee below
kafkaBridgeThe configuration for the Kafka bridge.objectSee belowSee below
kafkaStateDetectorThe configuration for the Kafka state detector.objectSee belowSee below
metrics.enabledWhether to enable the anonymous metrics service or not.booltrue or falsetrue
mqttBridgeThe configuration for the MQTT bridge.objectSee belowSee below
serialNumberThe hostname of the device. Used by some microservices to identify the device.stringAnydefault
tulipconnectorThe configuration for the Tulip connector.objectSee belowSee below

Data model v2

The _000_commonConfig.datamodel_v2 section contains the configuration related to the UNS data model.

The following table lists the configuration options that can be set in the _000_commonConfig.datamodel_v2 section:

datamodel_v2 section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the UNS data model should be used.booltrue, falsetrue
bridgesList of data bridges to create.listSee belowSee below
database.nameThe name of the database to use for the data model v2stringAnyumh_v2
database.hostThe host of the database to use for the data model v2stringAnyunited-manufacturing-hub
grafana.dbreaderThe name of the Grafana read-only database userstringAnygrafanareader
grafana.dbpasswordThe password of the Grafana read-only database userstringAnychangeme
Bridges

The _000_commonConfig.datamodel_v2.bridges section contains a list of configuration options for the data bridge. Each item in the list represents a data bridge instance, and the following table lists the configuration options that can be set in each item:

bridges section parameters
ParameterDescriptionTypeAllowed valuesDefault
modeThe mode of the data bridge.stringmqtt-kafka, kafka-kafka, mqtt-mqttmqtt-kafka
brokerAThe address of the source broker. Can be either MQTT or Kafka, and must include the portstringValid addressunited-manufacturing-hub-mqtt:1883
brokerBThe address of the destination broker. Can be either MQTT or Kafka, and must include the portstringValid addressunited-manufacturing-hub-kafka:9092
topicThe topic to subscribe to. Can be in either MQTT or Kafka form. Wildcards (# for MQTT, .* for Kafka) are allowed in order to subscribe to multiple topicsstringAnyumh.v1..*
topicMergePointThe nth part of the topic to use as the message key. If the topic is umh/v1/acme/anytown/foo/bar/#, and this value is 5, then all the messages wil end up in the topic umh.v1.acme.anytown.foointGreater than 35
partitionsThe number of partitions to use for the destination topic. Only used if the destination broker is Kafka.intGreater than 06
replicationFactorThe replication factor to use for the destination topic. Only used if the destination broker is Kafka.intOdd integer1
mqttEnableTLSWhether to enable TLS for the MQTT connection. Only used with MQTT brokersbooltrue, falsefalse
mqttPasswordThe password to use for the MQTT connection. Only used with MQTT brokersstringAny""
messageLRUSizeThe size of the LRU cache used to avoid message looping. Only used with MQTT brokersintAny1000000

Data sources

The _000_commonConfig.datasources section contains the configuration of the microservices used to acquire data, like the ones that connect to a sensor or simulate data.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources section:

datasources section parameters
ParameterDescriptionTypeAllowed valuesDefault
barcodereaderThe configuration of the barcodereader microservice.objectSee belowSee below
iotsensorsmqttThe configuration of the IoTSensorsMQTT microservice.objectSee belowSee below
opcuasimulatorThe configuration of the opcuasimulator microservice.objectSee belowSee below
packmlmqttsimulatorThe configuration of the packmlsimulator microservice.objectSee belowSee below
sensorconnectThe configuration of the sensorconnect microservice.objectSee belowSee below
Barcode reader

The _000_commonConfig.datasources.barcodereader section contains the configuration of the barcodereader microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.barcodereader section:

barcodereader section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the barcodereader microservice is enabled.booltrue, falsefalse
USBDeviceNameThe name of the USB device to use.stringAnyDatalogic ADC, Inc. Handheld Barcode Scanner
USBDevicePathThe path of the USB device to use. It is recommended to use a wildcard (for example, /dev/input/event*) or leave emptystringValid Unix device path""
customerIDThe customer ID to use in the topic structure.stringAnyraw
locationThe location to use in the topic structure.stringAnybarcodereader
machineIDThe asset ID to use in the topic structure.stringAnybarcodereader
IoT Sensors MQTT

The _000_commonConfig.datasources.iotsensorsmqtt section contains the configuration of the IoTSensorsMQTT microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.iotsensorsmqtt section:

iotsensorsmqtt section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the IoTSensorsMQTT microservice is enabled.booltrue, falsetrue
OPC UA Simulator

The _000_commonConfig.datasources.opcuasimulator section contains the configuration of the opcuasimulator microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.opcuasimulator section:

opcuasimulator section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the opcuasimulator microservice is enabled.booltrue, falsetrue
PackML MQTT Simulator

The _000_commonConfig.datasources.packmlmqttsimulator section contains the configuration of the packmlsimulator microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.packmlmqttsimulator section:

packmlmqttsimulator section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the packmlsimulator microservice is enabled.booltrue, falsetrue
Sensor connect

The _000_commonConfig.datasources.sensorconnect section contains the configuration of the sensorconnect microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.sensorconnect section:

sensorconnect section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the sensorconnect microservice is enabled.booltrue, falsefalse
iprangeThe IP range of the sensors in CIDR notation.stringValid IP range192.168.10.1/24
enableKafkaWhether the sensorconnect microservice should use Kafka.booltrue, falsetrue
enableMQTTWhether the sensorconnect microservice should use MQTT.booltrue, falsefalse
testModeWhether to enable test mode. Only useful for development.booltrue, falsefalse

Data processing

The _000_commonConfig.dataprocessing section contains the configuration of the microservices used to process data, such as the nodered microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.dataprocessing section:

dataprocessing section parameters
ParameterDescriptionTypeAllowed valuesDefault
noderedThe configuration of the nodered microservice.objectSee belowSee below
Node-RED

The _000_commonConfig.dataprocessing.nodered section contains the configuration of the nodered microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.dataprocessing.nodered section:

nodered section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the nodered microservice is enabled.booltrue, falsetrue
defaultFlowsWhether the default flows should be used.booltrue, falsefalse

Infrastructure

The _000_commonConfig.infrastructure section contains the configuration of the microservices responsible for connecting all the other microservices, such as the MQTT broker and the Kafka broker.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure section:

infrastructure section parameters
ParameterDescriptionTypeAllowed valuesDefault
mqttThe configuration of the MQTT broker.objectSee belowSee below
kafkaThe configuration of the Kafka broker.objectSee belowSee below
MQTT

The _000_commonConfig.infrastructure.mqtt section contains the configuration of the MQTT broker.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure.mqtt section:

mqtt section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the MQTT broker is enabledbooltrue, falsetrue
adminUser.enabledWhether the admin user is enabledbooltrue, falsefalse
adminUser.nameThe name of the admin userstringAny UTF-8 stringadmin-user
adminUser.encryptedPasswordThe encrypted password of the admin userstringAny""
tls.useTLSWhether TLS should be usedbooltrue, falsetrue
tls.insecureSkipVerifyWhether the SSL certificate validation should be skippedbooltrue, falsetrue
tls.keystoreBase64The base64 encoded keystorestringAny""
tls.keystorePasswordThe password of the keystorestringAny""
tls.truststoreBase64The base64 encoded truststorestringAny""
tls.truststorePasswordThe password of the truststorestringAny""
tls.caCertThe CA certificatestringAny""
tls.factoryinput.certThe certificate used for the factoryinput microservicestringAny""
tls.factoryinput.keyThe key used for the factoryinput microservicestringAny""
tls.mqtt_kafka_bridge.certThe certificate used for the mqttkafkabridgestringAny""
tls.mqtt_kafka_bridge.keyThe key used for the mqttkafkabridgestringAny""
tls.mqtt_bridge.local_certThe certificate used for the local mqttbridge brokerstringAny""
tls.mqtt_bridge.local_keyThe key used for the local mqttbridge brokerstringAny""
tls.mqtt_bridge.remote_certThe certificate used for the remote mqttbridge brokerstringAny""
tls.mqtt_bridge.remote_keyThe key used for the remote mqttbridge brokerstringAny""
tls.sensorconnect.certThe certificate used for the sensorconnect microservicestringAny""
tls.sensorconnect.keyThe key used for the sensorconnect microservicestringAny""
tls.iotsensorsmqtt.certThe certificate used for the iotsensorsmqtt microservicestringAny""
tls.iotsensorsmqtt.keyThe key used for the iotsensorsmqtt microservicestringAny""
tls.packmlsimulator.certThe certificate used for the packmlsimulator microservicestringAny""
tls.packmlsimulator.keyThe key used for the packmlsimulator microservicestringAny""
tls.nodered.certThe certificate used for the nodered microservicestringAny""
tls.nodered.keyThe key used for the nodered microservicestringAny""
Kafka

The _000_commonConfig.infrastructure.kafka section contains the configuration of the Kafka broker and related services, like mqttkafkabridge, kafkatopostgresql and the Kafka console.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure.kafka section:

kafka section parameters
ParameterDescriptionTypeAllowed valuesDefault
enabledWhether the Kafka broker and related services are enabledbooltrue, falsetrue
useSSLWhether SSL should be usedbooltrue, falsetrue
defaultTopicsThe default topics that should be createdstringSemicolon separated list of valid Kafka topicsia.test.test.test.processValue;ia.test.test.test.count;umh.v1.kafka.newTopic
tls.CACertThe CA certificatestringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafka.certThe certificate used for the kafka brokerstringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafka.privkeyThe private key of the certificate for the Kafka brokerstringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.barcodereader.sslKeyPasswordThe encrypted password of the SSL key for the barcodereader microservice. If empty, no password is usedstringAny""
tls.barcodereader.sslKeyPemThe private key for the SSL certificate of the barcodereader microservicestringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.barcodereader.sslCertificatePemThe private SSL certificate for the barcodereader microservicestringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkabridge.sslKeyPasswordLocalThe encrypted password of the SSL key for the local mqttbridge broker. If empty, no password is usedstringAny""
tls.kafkabridge.sslKeyPemLocalThe private key for the SSL certificate of the local mqttbridge brokerstringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.kafkabridge.sslCertificatePemLocalThe private SSL certificate for the local mqttbridge brokerstringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkabridge.sslCACertRemoteThe CA certificate for the remote mqttbridge brokerstringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkabridge.sslCertificatePemRemoteThe private SSL certificate for the remote mqttbridge brokerstringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkabridge.sslKeyPasswordRemoteThe encrypted password of the SSL key for the remote mqttbridge broker. If empty, no password is usedstringAny""
tls.kafkabridge.sslKeyPemRemoteThe private key for the SSL certificate of the remote mqttbridge brokerstringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.kafkadebug.sslKeyPasswordThe encrypted password of the SSL key for the kafkadebug microservice. If empty, no password is usedstringAny""
tls.kafkadebug.sslKeyPemThe private key for the SSL certificate of the kafkadebug microservicestringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.kafkadebug.sslCertificatePemThe private SSL certificate for the kafkadebug microservicestringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkainit.sslKeyPasswordThe encrypted password of the SSL key for the kafkainit microservice. If empty, no password is usedstringAny""
tls.kafkainit.sslKeyPemThe private key for the SSL certificate of the kafkainit microservicestringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.kafkainit.sslCertificatePemThe private SSL certificate for the kafkainit microservicestringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkastatedetector.sslKeyPasswordThe encrypted password of the SSL key for the kafkastatedetector microservice. If empty, no password is usedstringAny""
tls.kafkastatedetector.sslKeyPemThe private key for the SSL certificate of the kafkastatedetector microservicestringAny—–BEGIN PRIVATE KEY—– … —–END PRIVATE KEY—–
tls.kafkastatedetector.sslCertificatePemThe private SSL certificate for the kafkastatedetector microservicestringAny—–BEGIN CERTIFICATE—– … —–END CERTIFICATE—–
tls.kafkatopostgresql.sslKeyPasswordThe encrypted password of the SSL key for the kafkatopostgresql microservice. If empty, no password is usedstringAny""
tls.kafkatopostgresql.sslKeyPemThe private key for the SSL certificate of the kafkatopostgresql microservicestring