Architecture

• 1: Helm Chart
• 2: Microservices
• 2.1: Core
• 2.1.1: Cache
• 2.1.2: Database
• 2.1.3: Factoryinsight
• 2.1.4: Grafana
• 2.1.5: Kafka Bridge
• 2.1.6: Kafka Broker
• 2.1.7: Kafka Console
• 2.1.8: Kafka to Postgresql
• 2.1.9: MQTT Bridge
• 2.1.10: MQTT Broker
• 2.1.11: MQTT Kafka Bridge
• 2.1.12: Node-RED
• 2.1.13: Sensorconnect
• 2.2: Community
• 2.2.1: Barcodereader
• 2.2.2: Factoryinput
• 2.2.3: Grafana Proxy
• 2.2.4: Kafka State Detector
• 2.2.5: MQTT Simulator
• 2.2.6: MQTT to Postgresql
• 2.2.7: OPCUA Simulator
• 2.2.8: PackML Simulator
• 2.2.9: Tulip Connector
• 2.3: Grafana Plugins
• 2.3.1: Umh Datasource V2
• 2.3.2: Umh Datasource
• 2.3.3: Factoryinput Panel
• 3: Datamodel
• 3.1: Messages
• 3.1.1: activity
• 3.1.2: addOrder
• 3.1.3: addParentToChild
• 3.1.4: addProduct
• 3.1.5: addShift
• 3.1.6: count
• 3.1.7: deleteShift
• 3.1.8: detectedAnomaly
• 3.1.9: endOrder
• 3.1.10: modifyProducedPieces
• 3.1.11: modifyState
• 3.1.12: processValue
• 3.1.13: processValueString
• 3.1.14: productTag
• 3.1.15: productTagString
• 3.1.16: recommendation
• 3.1.17: scrapCount
• 3.1.18: scrapUniqueProduct
• 3.1.19: startOrder
• 3.1.20: state
• 3.1.21: uniqueProduct
• 3.2: Database
• 3.2.1: assetTable
• 3.2.2: configurationTable
• 3.2.3: countTable
• 3.2.4: orderTable
• 3.2.5: processValueStringTable
• 3.2.6: processValueTable
• 3.2.7: productTable
• 3.2.8: recommendationTable
• 3.2.9: shiftTable
• 3.2.10: stateTable
• 3.2.11: uniqueProductTable
• 3.3: States
• 3.3.1: Active (10000-29999)
• 3.3.2: Unknown (30000-59999)
• 3.3.3: Material (60000-99999)
• 3.3.4: Process(100000-139999)
• 3.3.5: Operator (140000-159999)
• 3.3.6: Planning (160000-179999)
• 3.3.7: Technical (180000-229999)

The United Manufacturing Hub at its core is a Helm Chart for Kubernetes consisting of several microservices and open source 3rd party applications, such as Node-RED and Grafana. This Helm Chart can be deployed in various environments, from edge devices and virtual machines to managed Kubernetes offerings. In large-scale deployments, you find typically a combination out of all these deployment options.

In this chapter, we’ll explore the various microservices and applications that make up the United Manufacturing Hub, and how they work together to help you extract, contextualize, store, and visualize data from your shop floor.

flowchart subgraph UMH["United Manufacturing Hub"] style UMH fill:#47a0b5 subgraph UNS["Unified Namespace"] style UNS fill:#f4f4f4 kafka["Apache Kafka"] mqtt["HiveMQ"] console["Console"] kafka-bridge mqtt-kafka-bridge["mqtt-kafka-bridge"] click kafka "./microservices/core/kafka" click mqtt "./microservices/core/mqtt-broker" click console "./microservices/core/console" click kafka-bridge "./microservices/core/kafka-bridge" click mqtt-kafka-bridge "./microservices/core/mqtt-kafka-bridge" mqtt <-- MQTT --> mqtt-kafka-bridge <-- Kafka --> kafka kafka -- Kafka --> console end subgraph custom["Custom Microservices"] custom-microservice["A user provied custom microservice in the Helm Chart"] custom-application["A user provided custom application deployed as Kubernetes resources or as a Helm Chart"] click custom-microservice "./microservices/core/custom" end subgraph Historian style Historian fill:#f4f4f4 kafka-to-postgresql timescaledb[("TimescaleDB")] factoryinsight umh-datasource grafana["Grafana"] redis click kafka-to-postgresql "./microservices/core/kafka-to-postgresql" click timescaledb "./microservices/core/database" click factoryinsight "./microservices/core/factoryinsight" click grafana "./microservices/core/grafana" click redis "./microservices/core/redis" kafka -- Kafka ---> kafka-to-postgresql kafka-to-postgresql -- SQL --> timescaledb timescaledb -- SQL --> factoryinsight factoryinsight -- HTTP --> umh-datasource umh-datasource --Plugin--> grafana factoryinsight <--RESP--> redis kafka-to-postgresql <--RESP--> redis end subgraph Connectivity style Connectivity fill:#f4f4f4 nodered["Node-RED"] barcodereader sensorconnect click nodered "./microservices/core/node-red" click barcodereader "./microservices/community/barcodereader" click sensorconnect "./microservices/core/sensorconnect" nodered <-- Kafka --> kafka barcodereader -- Kafka --> kafka sensorconnect -- Kafka --> kafka end subgraph Simulators style Simulators fill:#f4f4f4 mqtt-simulator["IoT sensors simulator"] packml-simulator["PackML simulator"] opcua-simulator["OPC-UA simulator"] click mqtt-simulator "./microservices/community/mqtt-simulator" click packml-simulator "./microservices/community/packml-simulator" click opcua-simulator "./microservices/community/opcua-simulator" mqtt-simulator -- MQTT --> mqtt packml-simulator -- MQTT --> mqtt opcua-simulator -- OPC-UA --> nodered end end subgraph Datasources plc["PLCs"] other["Other systems on the shopfloor (MES, ERP, etc.)"] barcode["USB barcode reader"] ifm["IO-link sensor"] iot["IoT devices"] plc -- "Siemens S7, OPC-UA, Modbus, etc." --> nodered other -- " " ----> nodered ifm -- HTTP --> sensorconnect barcode -- USB --> barcodereader iot <-- MQTT --> mqtt %% at the end for styling purposes nodered <-- MQTT --> mqtt end subgraph Data sinks umh-other["Other UMH instances"] other-systems["Other systems (cloud analytics, cold storage, BI tools, etc.)"] kafka <-- Kafka --> kafka-bridge kafka-bridge <-- Kafka ----> umh-other factoryinsight -- HTTP ----> other-systems end

Simulators

The United Manufacturing Hub includes several simulators to generate data during development and testing.

Microservices

• iotsensorsmqtt simulates data in three different MQTT topics, providing a simple way to test and visualize MQTT data streams.
• packml-simulator simulates a PackML machine which sends and receives MQTT messages
• opcua-simulator simulates an OPC-UA server, which can then be used to test connectivity of OPC-UA clients and to generate sample data for OPC-UA clients

Data connectivity microservices

The United Manufacturing Hub includes microservices that extract data from the shop floor and push it into the Unified Namespace. Additionally, you can deploy your own microservices or third-party solutions directly into the Kubernetes cluster using the custom microservice feature. To learn more about third-party solutions, check out our extensive tutorials on our learning hub

Microservices

• sensorconnect automatically reads out IO-Link Master and their connected sensors, and pushes the data to the message broker.
• barcodereader connects to USB barcode reader devices and pushes the data to the message broker.
• Node-RED is a versatile tool with many community plugins and allows access to machine PLCs or connections with other systems on the shopfloor. It plays an important role and is explained in the next section.

Node-RED: connectivity & contextualization

Node-RED is not just a tool for connectivity, but also for stream processing and data contextualization. It is often used to extract data from the message broker, reformat the event, and push it back into a different topic, such as the UMH datamodel.

In addition to the built-in microservices, third-party contextualization solutions can be deployed similarly to data connectivity microservices. For more information on these solutions, check out our extensive tutorials on our learning hub. In addition to the built-in microservices, third-party contextualization solutions can be deployed similarly to data connectivity microservices. For more information on these solutions, check out our extensive tutorials on our learning hub.

Microservices

• Node-RED is a programming tool that can wire together hardware devices, APIs, and online services.

Unified Namespace

At the core of the United Manufacturing Hub lies the Unified Namespace, which serves as the central source of truth for all events and messages occurring on your shop floor. The Unified Namespace is implemented using two message brokers: HiveMQ for MQTT and Apache Kafka. MQTT is used to receive data from IoT devices on the shop floor because it excels at handling a large number of unreliable connections. On the other hand, Kafka is used to enable communication between the microservices, leveraging its large-scale data processing capabilities.

The data between both brokers is bridged automatically using the mqtt-to-kafka microservice, allowing you to send data to MQTT and process it reliably in Kafka.

If you’re curious about the benefits of this dual approach to MQTT/Kafka, check out our blog article about Tools & Techniques for Scalable Dataprocessing in Industrial IoT.

For more information on the Unified Namespace feature and how to use it, check out the detailed description of the Unified Namespace feature.

Microservices

• HiveMQ is an MQTT broker used for receiving data from IoT devices on the shop floor. It excels at handling large numbers of unreliable connections.
• Apache Kafka is a distributed streaming platform used for communication between microservices. It offers large-scale data processing capabilities.
• mqtt-kafka-bridge is a microservice that bridges messages between MQTT and Kafka, allowing you to send data to MQTT and process them reliably in Kafka.
• kafka-bridge a microservice that bridges messages between multiple Kafka instances.
• console is a web-based user interface for Kafka, which provides a graphical view of topics and messages.

Historian / data storage and visualization

The United Manufacturing Hub stores events according to our datamodel. These events are automatically stored in TimescaleDB, an open-source time-series SQL database. From there, you can access the stored data using Grafana, a visualization and analytics software. With Grafana, you can perform on-the-fly data analysis by executing simple min, max, and avg on tags, or extended KPI calculations such as OEE. These calculations can be selected in the umh-datasource microservice.

For more information on the Historian or Analytics feature and how to use it, check out the detailed description of the Historian feature or the Analytics features.

Microservices

• kafka-to-postgresql stores data in selected topics from the Kafka broker in a PostgreSQL compatible database such as TimescaleDB.
• TimescaleDB, which is an open-source time-series SQL database
• factoryinsight provides REST endpoints to fetch data and calculate KPIs
• Grafana is a visualization and analytics software
• umh-datasource is a plugin providing access factoryinsight
• redis is an in-memory data structure store, used for cache.

Custom Microservices

The Helm Chart allows you to add your own microservices or Docker containers to the United Manufacturing Hub. These can be used, for example, to connect with third-party systems or to analyze the data. Additionally, you can deploy any other third-party application as long as it is available as a Helm Chart, Kubernetes resource, or Docker Compose (which can be converted to Kubernetes resources).

1 - Helm Chart

An Helm chart is a package manager for Kubernetes that simplifies the installation, configuration, and deployment of applications and services. It contains all the necessary Kubernetes manifests, configuration files, and dependencies required to run a particular application or service. One of the main advantages of Helm is that it allows to define the configuration of the installed resources in a single YAML file, called values.yaml. Helm provides great documentation on how to acheive this at https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing

The Helm Chart of the United Manufacturing Hub is composed of both custom microservices and third-party applications. If you want a more in-depth view of the architecture of the United Manufacturing Hub, you can read the Architecture overview page.

Helm Chart structure

Custom microservices

The Helm Chart of the United Manufacturing Hub is composed of the following custom microservices:

• barcodereader: reads the input from a barcode reader and sends it to the MQTT broker for further processing.
• customMicroservice: a template for deploying any number of custom microservices.
• factoryinput: provides REST endpoints for MQTT messages.
• factoryinsight: provides REST endpoints to fetch data and calculate KPIs.
• grafanaproxy: provides a proxy to the backend services.
• MQTT Simulator: simulates sensors and sends the data to the MQTT broker for further processing.
• kafka-bridge: connects Kafka brokers on different Kubernetes clusters.
• kafkatopostgresql: stores the data from the Kafka broker in a PostgreSQL database.
• mqtt-kafka-bridge: connects the MQTT broker and the Kafka broker.
• mqttbridge: connects MQTT brokers on different Kubernetes clusters.
• opcuasimulator: simulates OPC UA servers and sends the data to the MQTT broker for further processing.
• packmlmqttsimulator: simulates a PackML state machine and sends the data to the MQTT broker for further processing.
• sensorconnect: connects to a sensor and sends the data to the MQTT and Kafka brokers for further processing.
• tulip-connector: exposes internal APIs to the internet, especially tailored for the Tulip platform.

Third-party applications

The Helm Chart of the United Manufacturing Hub is composed of the following third-party applications:

• Grafana: a visualization and analytics software.
• HiveMQ: an MQTT broker.
• Node-RED: a programming tool for wiring together hardware devices, APIs and online services.
• Redis: an in-memory data structure store, used for cache.
• RedPanda: a Kafka-compatible distributed event streaming platform.
• RedPanda Console: a web-based user interface for RedPanda.
• TimescaleDB: an open-source time-series SQL database.

Configuration options

The Helm Chart of the United Manufacturing Hub can be configured by setting values in the values.yaml file. This file has three main sections that can be used to configure the applications:

• customers: contains the definition of the customers that will be created during the installation of the Helm Chart. This section is optional, and it’s used only by factoryinsight and factoryinput.
• _000_commonConfig: contains the basic configuration options to customize the United Manufacturing Hub, and it’s divided into sections that group applications with similar scope, like the ones that compose the infrastructure or the ones responsible for data processing. This is the section that should be mostly used to configure the microservices.
• _001_customMicroservices: used to define the configuration of custom microservices that are not included in the Helm Chart.

After those three sections, there are the specific sections for each microservice, which contain their advanced configuration. This is the so called Danger Zone, because the values in those sections should not be changed, unlsess you absolutely know what you are doing.

When a parameter contains . (dot) characters, it means that it is a nested parameter. For example, in the tls.factoryinput.cert parameter the cert parameter is nested inside the tls.factoryinput section, and the factoryinput section is nested inside the tls section.

Customers

The customers section contains the definition of the customers that will be created during the installation of the Helm Chart. It’s a simple dictionary where the key is the name of the customer, and the value is the password.

For example, the following snippet creates two customers:

customers:
  customer1: password1
  customer2: password2

Common configuration options

The _000_commonConfig contains the basic configuration options to customize the United Manufacturing Hub, and it’s divided into sections that group applications with similar scope.

The following table lists the configuration options that can be set in the _000_commonConfig section:

Data sources

The _000_commonConfig.datasources section contains the configuration of the microservices used to acquire data, like the ones that connect to a sensor or simulate data.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources section:

Barcode reader

The _000_commonConfig.datasources.barcodereader section contains the configuration of the barcodereader microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.barcodereader section:

IoT Sensors MQTT

The _000_commonConfig.datasources.iotsensorsmqtt section contains the configuration of the IoTSensorsMQTT microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.iotsensorsmqtt section:

OPC UA Simulator

The _000_commonConfig.datasources.opcuasimulator section contains the configuration of the opcuasimulator microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.opcuasimulator section:

PackML MQTT Simulator

The _000_commonConfig.datasources.packmlmqttsimulator section contains the configuration of the packmlsimulator microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.packmlmqttsimulator section:

Sensor connect

The _000_commonConfig.datasources.sensorconnect section contains the configuration of the sensorconnect microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.datasources.sensorconnect section:

Data processing

The _000_commonConfig.dataprocessing section contains the configuration of the microservices used to process data, such as the nodered microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.dataprocessing section:

Node-RED

The _000_commonConfig.dataprocessing.nodered section contains the configuration of the nodered microservice.

The following table lists the configuration options that can be set in the _000_commonConfig.dataprocessing.nodered section:

Infrastructure

The _000_commonConfig.infrastructure section contains the configuration of the microservices responsible for connecting all the other microservices, such as the MQTT broker and the Kafka broker.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure section:

MQTT

The _000_commonConfig.infrastructure.mqtt section contains the configuration of the MQTT broker.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure.mqtt section:

Kafka

The _000_commonConfig.infrastructure.kafka section contains the configuration of the Kafka broker and related services, like mqttkafkabridge, kafkatopostgresql and the Kafka console.

The following table lists the configuration options that can be set in the _000_commonConfig.infrastructure.kafka section:

Data storage

The _000_commonConfig.datastorage section contains the configuration of the microservices used to store data. Specifically, it controls the following microservices:

• cache
• database
• factoryinsight
• grafana
• kafka-to-postgresql
• mqtt-broker

If you want to specifically configure one of these microservices, you can do so in their respective sections in the Danger Zone.

The following table lists the configurable parameters of the _000_commonConfig.datastorage section.

Data input

The _000_commonConfig.datainput section contains the configuration of the microservices used to input data. Specifically, it controls the following microservices:

• factoryinput
• grafanaproxy

If you want to specifically configure one of these microservices, you can do so in their respective sections in the danger zone.

The following table lists the configurable parameters of the _000_commonConfig.datainput section./

MQTT Bridge

The _000_commonConfig.mqttBridge section contains the configuration of the mqtt-bridge microservice, responsible for bridging MQTT brokers in different Kubernetes clusters.

The following table lists the configurable parameters of the _000_commonConfig.mqttBridge section.

Kafka Bridge

The _000_commonConfig.kafkaBridge section contains the configuration of the kafka-bridge microservice, responsible for bridging Kafka brokers in different Kubernetes clusters.

The following table lists the configurable parameters of the _000_commonConfig.kafkaBridge section.

Topic Map

The topicmap parameter is a list of topic maps, each of which contains the following parameters:

For more information about the topic maps, see the kafka-bridge documentation.

Kafka State Detector

The _000_commonConfig.kafkaStateDetector section contains the configuration of the kafka-state-detector microservice, responsible for detecting the state of the Kafka broker.

The following table lists the configurable parameters of the _000_commonConfig.kafkaStateDetector section.

Debug

The _000_commonConfig.debug section contains the debug configuration for all the microservices. This values should not be enabled in production.

The following table lists the configurable parameters of the _000_commonConfig.debug section.

Tulip Connector

The _000_commonConfig.tulipconnector section contains the configuration of the tulip-connector microservice, responsible for connecting a Tulip instance with the United Manufacturing Hub.

The following table lists the configurable parameters of the _000_commonConfig.tulipconnector section.

Custom microservices configuration

The _001_customConfig section contains a list of custom microservices definitions. It can be used to deploy any application of your choice, which can be configured using the following parameters:

Danger zone

The next sections contain a more advanced configuration of the microservices. Usually, changing the values of the previous sections is enough to run the United Manufacturing Hub. However, you may need to adjust some of the values below if you want to change the default behavior of the microservices.

Everything below this point should not be changed, unless you know what you are doing.

Sections

barcodereader

The barcodereader section contains the advanced configuration of the barcodereader microservice.

factoryinput

The factoryinput section contains the advanced configuration of the factoryinput microservice.

env

The env section contains the configuration of the environment variables to add to the Pod.

factoryinsight

The factoryinsight section contains the advanced configuration of the factoryinsight microservice.

grafana

The grafana section contains the advanced configuration of the grafana microservice. This is based on the official Grafana Helm chart. For more information about the parameters, please refer to the official documentation.

Here are only the values different from the default ones.

datasources

The datasources section contains the configuration of the datasources provisioning. See the Grafana documentation for more information.

datasources.yaml:
  apiVersion: 1
  datasources:
    - name: umh-v2-datasource
      # <string, required> datasource type. Required
      type: umh-v2-datasource
      # <string, required> access mode. proxy or direct (Server or Browser in the UI). Required
      access: proxy
      # <int> org id. will default to orgId 1 if not specified
      orgId: 1
      url: "http://united-manufacturing-hub-factoryinsight-service/"
      jsonData:
        customerID: $FACTORYINSIGHT_CUSTOMERID
        apiKey: $FACTORYINSIGHT_PASSWORD
        baseURL: "http://united-manufacturing-hub-factoryinsight-service/"
        apiKeyConfigured: true
      version: 1
      # <bool> allow users to edit datasources from the UI.
      isDefault: false
      editable: false
    # <string, required> name of the datasource. Required
    - name: umh-datasource
      # <string, required> datasource type. Required
      type: umh-datasource
      # <string, required> access mode. proxy or direct (Server or Browser in the UI). Required
      access: proxy
      # <int> org id. will default to orgId 1 if not specified
      orgId: 1
      url: "http://united-manufacturing-hub-factoryinsight-service/"
      jsonData:
        customerId: $FACTORYINSIGHT_CUSTOMERID
        apiKey: $FACTORYINSIGHT_PASSWORD
        serverURL: "http://united-manufacturing-hub-factoryinsight-service/"
        apiKeyConfigured: true
      version: 1
      # <bool> allow users to edit datasources from the UI.
      isDefault: true
      editable: false
    # <string, required> name of the datasource. Required

envValueFrom

The envValueFrom section contains the configuration of the environment variables to add to the Pod, from a secret or a configmap.

env

The env section contains the configuration of the environment variables to add to the Pod.

extraInitContainers

The extraInitContainers section contains the configuration of the extra init containers to add to the Pod.

The init-plugins container is used to install the default plugins shipped with the UMH version of Grafana without the need to have an internet connection. See the documentation for a list of the plugins.

- image: unitedmanufacturinghub/grafana-umh:1.2.0
  name: init-plugins
  imagePullPolicy: IfNotPresent
  command: ['sh', '-c', 'cp -r /plugins /var/lib/grafana/']
  volumeMounts:
    - name: storage
      mountPath: /var/lib/grafana

grafana.ini

The grafana.ini section contains the configuration of the grafana.ini file. See the Grafana documentation for more information.

paths:
  data: /var/lib/grafana/data
  logs: /var/log/grafana
  plugins: /var/lib/grafana/plugins
  provisioning: /etc/grafana/provisioning
database:
  host: united-manufacturing-hub
  user: "grafana"
  name: "grafana"
  password: "changeme"
  ssl_mode: require
  type: postgres

grafanaproxy

The grafanaproxy section contains the configuration of the Grafana proxy microservice.

iotsensorsmqtt

The iotsensorsmqtt section contains the configuration of the IoT Sensors MQTT microservice.

kafkabridge

The kafkabridge section contains the configuration of the Kafka bridge.

kafkastatedetector

The kafkastatedetector section contains the configuration of the Kafka state detector.

kafkatopostgresql

The kafkatopostgresql section contains the configuration of the Kafka to PostgreSQL microservice.

metrics

The metrics section contains the configuration of the metrics CronJob that sends anonymous usage data.

mqtt_broker

The mqtt_broker section contains the configuration of the MQTT broker.

initContainer

The initContainer section contains the configuration for the init containers. By default, the hivemqextensioninit container is used to initialize the HiveMQ extensions.

initContainer:
  hivemqextensioninit:
    image:
      repository: unitedmanufacturinghub/hivemq-init
      tag: 2.0.0
      pullPolicy: IfNotPresent

mqttbridge

The mqttbridge section contains the configuration of the MQTT bridge.

mqttkafkabridge

The mqttkafkabridge section contains the configuration of the MQTT-Kafka bridge.

nodered

The nodered section contains the configuration of the Node-RED microservice.

env

The env section contains the environment variables to add to the Pod.

opcuasimulator

The opcuasimulator section contains the configuration of the OPC UA Simulator microservice.

packmlmqttsimulator

The packmlmqttsimulator section contains the configuration of the PackML MQTT Simulator microservice.

env

The env section contains the environment variables to add to the Pod.

redis

The redis section contains the configuration of the Redis microservice. This is based on the official Redis Helm chart. For more information about the parameters, see the official documentation.

Here are only the values different from the default ones.

commonConfiguration

The commonConfiguration section contains the common configuration to be added into the ConfigMap. For more information, see the documentation.

# Enable AOF https://redis.io/topics/persistence#append-only-file
appendonly yes
# Disable RDB persistence, AOF persistence already enabled.
save ""
# Backwards compatability with Redis version 6.*
replica-ignore-disk-write-errors yes

redpanda

The redpanda section contains the configuration of the Kafka broker. This is based on the RedPanda chart. For more information about the parameters, see the official documentation.

Here are only the values different from the default ones.

console

The console section contains the configuration of the RedPanda Console.

For more information about the parameters, see the official documentation.

sensorconnect

The sensorconnect section contains the configuration of the Sensorconnect microservice.

serviceAccount

The serviceAccount section contains the configuration of the service account. See the Kubernetes documentation for more information.

timescaledb-single

The timescaledb-single section contains the configuration of the TimescaleDB microservice. This is based on the official TimescaleDB Helm chart. For more information about the parameters, see the official documentation.

Here are only the values different from the default ones.

postInit

The postInit parameter is a list of references to sources that contain post init scripts. The scripts are executed after the database is initialized.

postInit:
  - configMap:
      name: {{ resource type="configmap" name="database" }}
      optional: false
  - secret:
      name: {{ resource type="secret" name="database" }}
      optional: false

tulipconnector

The tulipconnector section contains the configuration of the Tulip Connector microservice.

env

The env section contains the configuration of the environment variables to add to the Pod.

What’s next

• Take a look at the overview of the architecture
• Deep dive into the microservices

2 - Microservices

There are several microservices that are part of the United Manufacturing Hub. Some of them compose the core of the platform, and are mainly developed by the UMH team, with the addition of some third-party software. Others are maintained by the community, and are used to extend the functionality of the platform.

2.1 - Core

The microservices in this section are part of the Core of the United Manufacturing Hub. They are mainly developed by the UMH team, with the addition of some third-party software. They are used to provide the core functionality of the platform.

2.1.1 - Cache

The cache in the United Manufacturing Hub is Redis, a key-value store that is used as a cache for the other microservices.

How it works

Recently used data is stored in the cache to reduce the load on the database. All the microservices that need to access the database will first check if the data is available in the cache. If it is, it will be used, otherwise the microservice will query the database and store the result in the cache.

By default, Redis is configured to run in standalone mode, which means that it will only have one master node.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-redis-master
• Service:
  • Internal ClusterIP:
    • Redis: united-manufacturing-hub-redis-master at port 6379
    • Headless: united-manufacturing-hub-redis-headless at port 6379
    • Metrics: united-manufacturing-hub-redis-metrics at port 6379
• ConfigMap:
  • Configuration: united-manufacturing-hub-redis-configuration
  • Health: united-manufacturing-hub-redis-health
  • Scripts: united-manufacturing-hub-redis-scripts
• Secret: redis-secret
• PersistentVolumeClaim: redis-data-united-manufacturing-hub-redis-master-0

Configuration

You shouldn’t need to configure the cache manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the redis section of the Helm chart values file.

You can consult the Bitnami Redis chart for more information about the available configuration options.

Environment variables

2.1.2 - Database

The database microservice is the central component of the United Manufacturing Hub and is based on TimescaleDB, an open-source relational database built for handling time-series data. TimescaleDB is designed to provide scalable and efficient storage, processing, and analysis of time-series data.

You can find more information on the datamodel of the database in the Data Model section, and read about the choice to use TimescaleDB in the blog article.

How it works

When deployed, the database microservice will create two databases, with the related usernames and passwords:

• grafana: This database is used by Grafana to store the dashboards and other data.
• factoryinsight: This database is the main database of the United Manufacturing Hub. It contains all the data that is collected by the microservices.

Then, it creates the tables based on the database schema.

If you want to learn more about how TimescaleDB works, you can read the TimescaleDB documentation.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-timescaledb
• Service:
  • Internal ClusterIP for the replicas: united-manufacturing-hub-replica at port 5432
  • Internal ClusterIP for the config: united-manufacturing-hub-config at port 8008
  • External LoadBalancer: united-manufacturing-hub at port 5432
• ConfigMap:
  • Patroni: united-manufacturing-hub-timescaledb-patroni
  • Post init: timescale-post-init
  • Postgres BackRest: united-manufacturing-hub-timescaledb-pgbackrest
  • Scripts: united-manufacturing-hub-timescaledb-scripts
• Secret:
  • Certificate: united-manufacturing-hub-certificate
  • Patroni credentials: united-manufacturing-hub-credentials
  • Users passwords: timescale-post-init-pw
• PersistentVolumeClaim:
  • Data: storage-volume-united-manufacturing-hub-timescaledb-0
  • WAL-E: wal-volume-united-manufacturing-hub-timescaledb-0

Configuration

There is only one parameter that usually needs to be changed: the password used to connect to the database. To do so, set the value of the db_password key in the _000_commonConfig.datastorage section of the Helm chart values file.

Environment variables

2.1.3 - Factoryinsight

Factoryinsight is a microservice that provides a set of REST APIs to access the data from the database. It is particularly useful to calculate the Key Performance Indicators (KPIs) of the factories.

How it works

Factoryinsight exposes REST APIs to access the data from the database or calculate the KPIs. By default, it’s only accessible from the internal network of the cluster, but it can be configured to be accessible from the external network.

The APIs require authentication, that can be ehither a Basic Auth or a Bearer token. Both of these can be found in the Secret factoryinsight-secret.

API documentation

Kubernetes resources

• Deployment: united-manufacturing-hub-factoryinsight-deployment
• Service:
  • Internal ClusterIP: united-manufacturing-hub-factoryinsight-service at port 80
  • External : Access factoryinsight outside the cluster
• Secret: factoryinsight-secret

Configuration

You shouldn’t need to configure Factoryinsight manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the factoryinsight section of the Helm chart values file.

Environment variables

2.1.4 - Grafana

The grafana microservice is a web application that provides visualization and analytics capabilities. Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored.

It has a rich ecosystem of plugins that allow you to extend its functionality beyond the core features.

How it works

Grafana is a web application that can be accessed through a web browser. It let’s you create dashboards that can be used to visualize data from the database.

Thanks to some custom datasource plugins, Grafana can use the various APIs of the United Manufacturing Hub to query the database and display useful information.

Kubernetes resources

• Deployment: united-manufacturing-hub-grafana
• Service:
  • External LoadBalancer: united-manufacturing-hub-grafana at port 8080
• ConfigMap: united-manufacturing-hub-grafana
• Secret: grafana-secret
• PersistentVolumeClaim: united-manufacturing-hub-grafana

Configuration

Grafana is configured through its user interface. The default credentials are found in the grafana-secret Secret.

The Grafana installation that is provided by the United Manufacturing Hub is shipped with a set of preinstalled plugins:

• ACE.SVG by Andrew Rodgers
• Button Panel by CloudSpout LLC
• Button Panel by UMH Systems Gmbh
• Discrete by Natel Energy
• Dynamic Text by Marcus Olsson
• FlowCharting by agent
• Pareto Chart by isaozler
• Pie Chart (old) by Grafana Labs
• Timepicker Buttons Panel by williamvenner
• UMH Datasource by UMH Systems Gmbh
• UMH Datasource v2 by UMH Systems Gmbh
• Untimely by factry
• Worldmap Panel by Grafana Labs

Environment variables

2.1.5 - Kafka Bridge

Kafka-bridge is a microservice that connects two Kafka brokers and forwards messages between them. It is used to connect the local broker of the edge computer with the remote broker on the server.

How it works

This microservice has two ways of operation:

• High Integrity: This mode is used for topics that are critical for the user. It is garanteed that no messages are lost. This is achieved by committing the message only after it has been successfully inserted into the database. Ususally all the topics are forwarded in this mode, except for processValue, processValueString and raw messages.
• High Throughput: This mode is used for topics that are not critical for the user. They are forwarded as fast as possible, but it is possible that messages are lost, for example if the database struggles to keep up. Usually only the processValue, processValueString and raw messages are forwarded in this mode.

Kubernetes resources

• Deployment: united-manufacturing-hub-kafkabridge
• Secret:
  • Local broker: united-manufacturing-hub-kafkabridge-secrets-local
  • Remote broker: united-manufacturing-hub-kafkabridge-secrets-remote

Configuration

You can configure the kafka-bridge microservice by setting the following values in the _000_commonConfig.kafkaBridge section of the Helm chart values file.

  kafkaBridge:
    enabled: true
    remotebootstrapServer: ""
    topicmap:
      - bidirectional: false
        name: HighIntegrity
        send_direction: to_remote
        topic: ^ia\..+\..+\..+\.((addMaintenanceActivity)|(addOrder)|(addParentToChild)|(addProduct)|(addShift)|(count)|(deleteShiftByAssetIdAndBeginTimestamp)|(deleteShiftById)|(endOrder)|(modifyProducedPieces)|(modifyState)|(productTag)|(productTagString)|(recommendation)|(scrapCount)|(startOrder)|(state)|(uniqueProduct)|(scrapUniqueProduct))$
      - bidirectional: false
        name: HighThroughput
        send_direction: to_remote
        topic: ^ia\..+\..+\..+\.(processValue).*$

Topic Map schema

The topic map is a list of objects, each object represents a topic (or a set of topics) that should be forwarded. The following JSON schema describes the structure of the topic map:

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "type": "array",
    "title": "Kafka Topic Map",
    "description": "This schema validates valid Kafka topic maps.",
    "default": [],
    "additionalItems": true,
    "items": {
        "$id": "#/items",
        "anyOf": [
            {
                "$id": "#/items/anyOf/0",
                "type": "object",
                "title": "Unidirectional Kafka Topic Map with send direction",
                "description": "This schema validates entries, that are unidirectional and have a send direction.",
                "default": {},
                "examples": [
                    {
                        "name": "HighIntegrity",
                        "topic": "^ia\\..+\\..+\\..+\\.(?!processValue).+$",
                        "bidirectional": false,
                        "send_direction": "to_remote"
                    }
                ],
                "required": [
                    "name",
                    "topic",
                    "bidirectional",
                    "send_direction"
                ],
                "properties": {
                    "name": {
                        "$id": "#/items/anyOf/0/properties/name",
                        "type": "string",
                        "title": "Entry Name",
                        "description": "Name of the map entry, only used for logging & tracing.",
                        "default": "",
                        "examples": [
                            "HighIntegrity"
                        ]
                    },
                    "topic": {
                        "$id": "#/items/anyOf/0/properties/topic",
                        "type": "string",
                        "title": "The topic to listen on",
                        "description": "The topic to listen on, this can be a regular expression.",
                        "default": "",
                        "examples": [
                            "^ia\\..+\\..+\\..+\\.(?!processValue).+$"
                        ]
                    },
                    "bidirectional": {
                        "$id": "#/items/anyOf/0/properties/bidirectional",
                        "type": "boolean",
                        "title": "Is the transfer bidirectional?",
                        "description": "When set to true, the bridge will consume and produce from both brokers",
                        "default": false,
                        "examples": [
                            false
                        ]
                    },
                    "send_direction": {
                        "$id": "#/items/anyOf/0/properties/send_direction",
                        "type": "string",
                        "title": "Send direction",
                        "description": "Can be either 'to_remote' or 'to_local'",
                        "default": "",
                        "examples": [
                            "to_remote",
                            "to_local"
                        ]
                    }
                },
                "additionalProperties": true
            },
            {
                "$id": "#/items/anyOf/1",
                "type": "object",
                "title": "Bi-directional Kafka Topic Map with send direction",
                "description": "This schema validates entries, that are bi-directional.",
                "default": {},
                "examples": [
                    {
                        "name": "HighIntegrity",
                        "topic": "^ia\\..+\\..+\\..+\\.(?!processValue).+$",
                        "bidirectional": true
                    }
                ],
                "required": [
                    "name",
                    "topic",
                    "bidirectional"
                ],
                "properties": {
                    "name": {
                        "$id": "#/items/anyOf/1/properties/name",
                        "type": "string",
                        "title": "Entry Name",
                        "description": "Name of the map entry, only used for logging & tracing.",
                        "default": "",
                        "examples": [
                            "HighIntegrity"
                        ]
                    },
                    "topic": {
                        "$id": "#/items/anyOf/1/properties/topic",
                        "type": "string",
                        "title": "The topic to listen on",
                        "description": "The topic to listen on, this can be a regular expression.",
                        "default": "",
                        "examples": [
                            "^ia\\..+\\..+\\..+\\.(?!processValue).+$"
                        ]
                    },
                    "bidirectional": {
                        "$id": "#/items/anyOf/1/properties/bidirectional",
                        "type": "boolean",
                        "title": "Is the transfer bidirectional?",
                        "description": "When set to true, the bridge will consume and produce from both brokers",
                        "default": false,
                        "examples": [
                            true
                        ]
                    }
                },
                "additionalProperties": true
            }
        ]
    },
    "examples": [
   {
      "name":"HighIntegrity",
      "topic":"^ia\\..+\\..+\\..+\\.(?!processValue).+$",
      "bidirectional":true
   },
   {
      "name":"HighThroughput",
      "topic":"^ia\\..+\\..+\\..+\\.(processValue).*$",
      "bidirectional":false,
      "send_direction":"to_remote"
   }
]
}

Environment variables

2.1.6 - Kafka Broker

The Kafka broker in the United Manufacturing Hub is RedPanda, a Kafka-compatible event streaming platform. It’s used to store and process messages, in order to stream real-time data between the microservices.

How it works

RedPanda is a distributed system that is made up of a cluster of brokers, designed for maximum performance and reliability. It does not depend on external systems like ZooKeeper, as it’s shipped as a single binary.

Read more about RedPanda in the official documentation.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-kafka
• Service:
  • Internal ClusterIP (headless): united-manufacturing-hub-kafka
  • External NodePort: united-manufacturing-hub-kafka-external at port 9094 for the Kafka API listener, port 9644 for the Admin API listener, port 8083 for the HTTP Proxy listener, and port 8081 for the Schema Registry listener.
• ConfigMap: united-manufacturing-hub-kafka
• Secret: united-manufacturing-hub-kafka-sts-lifecycle
• PersistentVolumeClaim: datadir-united-manufacturing-hub-kafka-0

Configuration

You shouldn’t need to configure the Kafka broker manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the redpanda section of the Helm chart values file.

Environment variables

2.1.7 - Kafka Console

Kafka-console uses Redpanda Console to help you manage and debug your Kafka workloads effortlessy.

With it, you can explore your Kafka topics, view messages, list the active consumers, and more.

How it works

You can access the Kafka console via its Service.

It’s automatically connected to the Kafka broker, so you can start using it right away. You can view the Kafka broker configuration in the Broker tab, and explore the topics in the Topics tab.

Kubernetes resources

• Deployment: united-manufacturing-hub-console
• Service:
  • External LoadBalancer: united-manufacturing-hub-console at port 8090
• ConfigMap: united-manufacturing-hub-console
• Secret: united-manufacturing-hub-console

Configuration

Environment variables

2.1.8 - Kafka to Postgresql

Kafka-to-postgresql is a microservice responsible for consuming kafka messages and inserting the payload into a Postgresql database. Take a look at the Datamodel to see how the data is structured.

This microservice requires that the Kafka Topic umh.v1.kafka.newTopic exits. This will happen automatically from version 0.9.12.

How it works

By default, kafka-to-postgresql sets up two Kafka consumers, one for the High Integrity topics and one for the High Throughput topics.

The graphic below shows the program flow of the microservice.

High integrity

The High integrity topics are forwarded to the database in a synchronous way. This means that the microservice will wait for the database to respond with a non error message before committing the message to the Kafka broker. This way, the message is garanteed to be inserted into the database, even though it might take a while.

Most of the topics are forwarded in this mode.

The picture below shows the program flow of the high integrity mode.

High throughput

The High throughput topics are forwarded to the database in an asynchronous way. This means that the microservice will not wait for the database to respond with a non error message before committing the message to the Kafka broker. This way, the message is not garanteed to be inserted into the database, but the microservice will try to insert the message into the database as soon as possible. This mode is used for the topics that are expected to have a high throughput.

The topics that are forwarded in this mode are processValue, processValueString and all the raw topics.

Kubernetes resources

• Deployment: united-manufacturing-hub-kafkatopostgresql
• Secret: united-manufacturing-hub-kafkatopostgresql-certificates

Configuration

You shouldn’t need to configure kafka-to-postgresql manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the kafkatopostgresql section of the Helm chart values file.

Environment variables

2.1.9 - MQTT Bridge

MQTT-bridge is a microservice that connects two MQTT brokers and forwards messages between them. It is used to connect the local broker of the edge computer with the remote broker on the server.

How it works

This microservice subscribes to topics on the local broker and publishes the messages to the remote broker, while also subscribing to topics on the remote broker and publishing the messages to the local broker.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-mqttbridge
• Secret: united-manufacturing-hub-mqttbridge-secrets
• PersistentVolumeClaim: united-manufacturing-hub-mqttbridge-claim

Configuration

You can configure the URL of the remote MQTT broker that MQTT-bridge should connect to by setting the value of the remoteBrokerUrl parameter in the _000_commonConfig.mqttBridge section of the Helm chart values file.

Environment variables

2.1.10 - MQTT Broker

The MQTT broker in the United Manufacturing Hub is HiveMQ and is customized to fit the needs of the stack. It’s a core component of the stack and is used to communicate between the different microservices.

How it works

The MQTT broker is responsible for receiving MQTT messages from the different microservices and forwarding them to the MQTT Kafka bridge.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-hivemqce
• Service:
  • Internal ClusterIP:
    • HiveMQ local: united-manufacturing-hub-hivemq-local-service at port 1883 (MQTT) and 8883 (MQTT over TLS)
    • VerneMQ (for backwards compatibility): united-manufacturing-hub-vernemq at port 1883 (MQTT) and 8883 (MQTT over TLS)
    • VerneMQ local (for backwards compatibility): united-manufacturing-hub-vernemq-local-service at port 1883 (MQTT) and 8883 (MQTT over TLS)
  • External LoadBalancer: united-manufacturing-hub-mqtt at port 1883 (MQTT) and 8883 (MQTT over TLS)
• ConfigMap:
  • Configuration: united-manufacturing-hub-hivemqce-hive
  • Credentials: united-manufacturing-hub-hivemqce-extension
• Secret: united-manufacturing-hub-hivemqce-secret-keystore
• PersistentVolumeClaim:
  • Data: united-manufacturing-hub-hivemqce-claim-data
  • Extensions: united-manufacturing-hub-hivemqce-claim-extensions

Configuration

Most of the configuration is done through the XML files in the ConfigMaps. The default configuration should be sufficient for most use cases.

The HiveMQ installation of the United Manufacturing Hub comes with these extensions:

• RBAC file extension to manage the authentication and authorizations rules for the broker.
• Prometheus extension to expose metrics for a prometheus applications
• Heartbeat extension to allow for readiness checks

If you want to add more extensions, or to change the configuration, visit the HiveMQ documentation.

Environment variables

2.1.11 - MQTT Kafka Bridge

Mqtt-kafka-bridge is a microservice that acts as a bridge between MQTT brokers and Kafka brokers, transfering messages from one to the other and vice versa.

This microservice requires that the Kafka Topic umh.v1.kafka.newTopic exits. This will happen automatically from version 0.9.12.

Since version 0.9.10, it allows all raw messages, even if their content is not in a valid JSON format.

How it works

Mqtt-kafka-bridge consumes topics from a message broker, translates them to the proper format and publishes them to the other message broker.

Kubernetes resources

• Deployment: united-manufacturing-hub-mqttkafkabridge
• Secret:
  • Kafka: united-manufacturing-hub-mqttkafkabridge-kafka-secrets
  • MQTT: united-manufacturing-hub-mqttkafkabridge-mqtt-secrets

Configuration

You shouldn’t need to configure mqtt-kafka-bridge manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the mqttkafkabridge section of the Helm chart values file.

Environment variables

2.1.12 - Node-RED

Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the Node-RED library.

How it works

Node-RED is a JavaScript-based tool that can be used to create flows that interact with the other microservices in the United Manufacturing Hub or external services.

See our guides for Node-RED to learn more about how to use it.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-nodered
• Service:
  • External LoadBalancer: united-manufacturing-hub-nodered-service at port 1880
• ConfigMap:
  • Configuration: united-manufacturing-hub-nodered-config
  • Flows: united-manufacturing-hub-nodered-flows
• Secret: united-manufacturing-hub-nodered-secrets
• PersistentVolumeClaim: united-manufacturing-hub-nodered-claim

Configuration

You can enable the nodered microservice and decide if you want to use the default flows in the _000_commonConfig.dataprocessing.nodered section of the Helm chart values.

All the other values are set by default and you can find them in the Danger Zone section of the Helm chart values.

Environment variables

2.1.13 - Sensorconnect

Sensorconnect automatically detects ifm gateways connected to the network and reads data from the connected IO-Link sensors.

How it works

Sensorconnect continuosly scans the given IP range for gateways, making it effectively a plug-and-play solution. Once a gateway is found, it automatically download the IODD files for the connected sensors and starts reading the data at the configured interval. Then it processes the data and sends it to the MQTT or Kafka broker, to be consumed by other microservices.

If you want to learn more about how to use sensors in your asstes, check out the retrofitting section of the UMH Learn website.

IODD files

The IODD files are used to describe the sensors connected to the gateway. They contain information about the data type, the unit of measurement, the minimum and maximum values, etc. The IODD files are downloaded automatically from IODDFinder once a sensor is found, and are stored in a Persistent Volume. If downloading from internet is not possible, for example in a closed network, you can download the IODD files manually and store them in the folder specified by the IODD_FILE_PATH environment variable.

If no IODD file is found for a sensor, the data will not be processed, but sent to the broker as-is.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-sensorconnect
• Secret:
  • Kafka: united-manufacturing-hub-sensorconnect-kafka-secrets
  • MQTT: united-manufacturing-hub-sensorconnect-mqtt-secrets
• PersistentVolumeClaim: united-manufacturing-hub-sensorconnect-claim

Configuration

You can configure the IP range to scan for gateways, and which message broker to use, by setting the values of the parameters in the _000_commonConfig.datasources.sensorconnect section of the Helm chart values file.

The default values of the other parameters are usually good for most use cases, but you can change them in the Danger Zone section of the Helm chart values file.

Environment variables

Slowdown map

The ADDITIONAL_SLOWDOWN_MAP environment variable allows you to slow down and speed up the polling time of specific sensors. It is a JSON array of values, with the following structure:

[
  {
    "serialnumber": "000200610104",
    "slowdown_ms": -10
  },
  {
    "url": "http://192.168.0.13",
    "slowdown_ms": 20
  },
  {
    "productcode": "AL13500",
    "slowdown_ms": 20.01
  }
]

2.2 - Community

The microservices in this section are not part of the Core of the United Manufacturing Hub, either because they are still in development, deprecated or only supported community. They can be used to extend the functionality of the platform.

It is not recommended to use these microservices in production as they might be unstable or not supported anymore.

2.2.1 - Barcodereader

This microservice is still in development and is not considered stable for production use.

Barcodereader is a microservice that reads barcodes and sends the data to the Kafka broker.

How it works

Connect a barcode scanner to the system and the microservice will read the barcodes and send the data to the Kafka broker.

Kubernetes resources

• Deployment: united-manufacturing-hub-barcodereader
• Secret: united-manufacturing-hub-barcodereader-secrets

Configuration

Environment variables

2.2.2 - Factoryinput

This microservice is still in development and is not considered stable for production use

Factoryinput provides REST endpoints for MQTT messages via HTTP requests.

This microservice is typically accessed via grafana-proxy

How it works

The factoryinput microservice provides REST endpoints for MQTT messages via HTTP requests.

The main endpoint is /api/v1/{customer}/{location}/{asset}/{value}, with a POST request method. The customer, location, asset and value are all strings. And are used to build the MQTT topic. The body of the HTTP request is used as the MQTT payload.

Kubernetes resources

• StatefulSet: united-manufacturing-hub-factoryinput
• Service:
  • Internal ClusterIP: united-manufacturing-hub-factoryinput-service at port 80
• Secret: factoryinput-secret

Configuration

Environment variables

2.2.3 - Grafana Proxy

This microservice is still in development and is not considered stable for production use

How it works

The grafana-proxy microservice serves an HTTP REST endpoint located at /api/v1/{service}/{data}. The service parameter specifies the backend service to which the request should be proxied, like factoryinput or factoryinsight. The data parameter specifies the API endpoint to forward to the backend service. The body of the HTTP request is used as the payload for the proxied request.

Kubernetes resources

• Deployment: united-manufacturing-hub-grafanaproxy
• Service:
  • External LoadBalancer: united-manufacturing-hub-grafanaproxy-service at port 2096

Configuration

Environment variables

2.2.4 - Kafka State Detector

This microservice is still in development and is not considered stable for production use

How it works

Kubernetes resources

• Deployment: united-manufacturing-hub-kafkastatedetector
• Secret: united-manufacturing-hub-kafkastatedetector-secrets

Configuration

Environment variables

2.2.5 - MQTT Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but is enabled by default.

The IoTSensors MQTT Simulator is a microservice that simulates sensors sending data to the MQTT broker. You can read the full documentation on the

GitHub repository.

How it works

The microservice publishes messages on the topic ia/raw/development/ioTSensors/, creating a subtopic for each simulation. The subtopics are the names of the simulations, which are Temperature, Humidity, and Pressure. The values are calculated using a normal distribution with a mean and standard deviation that can be configured.

Kubernetes resources

• Deployment: united-manufacturing-hub-iotsensorsmqtt
• ConfigMap: united-manufacturing-hub-iotsensors-mqtt

Configuration

You can change the configuration of the microservice by updating the config.json file in the ConfigMap.

2.2.6 - MQTT to Postgresql

If you landed here from Google, you probably might want to check out either the architecture of the United Manufacturing Hub or our knowledge website for more information on the general topics of IT, OT and IIoT.

This microservice is deprecated and should not be used anymore in production. Please use kafka-to-postgresql instead.

How it works

The mqtt-to-postgresql microservice subscribes to the MQTT broker and saves the values of the messages on the topic ia/# in the database.

2.2.7 - OPCUA Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but is enabled by default.

How it works

The OPCUA Simulator is a microservice that simulates OPCUA devices. You can read the full documentation on the GitHub repository.

You can then connect to the simulated OPCUA server via Node-RED and read the values of the simulated devices. Learn more about how to connect to the OPCUA simulator to Node-RED in our guide.

Kubernetes resources

• Deployment: united-manufacturing-hub-opcuasimulator-deployment
• Service:
  • External LoadBalancer: united-manufacturing-hub-opcuasimulator-service at port 46010
• ConfigMap: united-manufacturing-hub-opcuasimulator-config

Configuration

You can change the configuration of the microservice by updating the config.json file in the ConfigMap.

2.2.8 - PackML Simulator

This microservice is a community contribution and is not part of the main stack of the United Manufacturing Hub, but it is enabled by default.

PackML MQTT Simulator is a virtual line that interfaces using PackML implemented over MQTT. It implements the following PackML State model and communicates over MQTT topics as defined by environmental variables. The simulator can run with either a basic MQTT topic structure or SparkPlugB.

How it works

You can read the full documentation on the GitHub repository.

Kubernetes resources

• Deployment: united-manufacturing-hub-packmlmqttsimulator

Configuration

You shouldn’t need to configure PackML Simulator manually, as it’s configured automatically when the cluster is deployed. However, if you need to change the configuration, you can do it by editing the packmlmqttsimulator section of the Helm chart values file.

Environment variables

2.2.9 - Tulip Connector

This microservice is still in development and is not considered stable for production use.

The tulip-connector microservice enables communication with the United Manufacturing Hub by exposing internal APIs, like factoryinsight, to the internet. With this REST endpoint, users can access data stored in the UMH and seamlessly integrate Tulip with a Unified Namespace and on-premise Historian. Furthermore, the tulip-connector can be customized to meet specific customer requirements, including integration with an on-premise MES system.

How it works

The tulip-connector acts as a proxy between the internet and the UMH. It exposes an endpoint to forward requests to the UMH and returns the response.

API documentation

Kubernetes resources

• Deployment: united-manufacturing-hub-tulip-connector-deployment
• Service:
  • Internal ClusterIP: united-manufacturing-hub-tulip-connector-service at port 80
• Ingress: united-manufacturing-hub-tulip-connector-ingress

Configuration

You can enable the tulip-connector and set the domain for the ingress by editing the values in the _000_commonConfig.tulipconnector section of the Helm chart values file.

Environment variables

2.3 - Grafana Plugins

2.3.1 - Umh Datasource V2

The plugin, umh-datasource-v2, is a Grafana data source plugin that allows you to fetch resources from a database and build queries for your dashboard.

How it works

1. When creating a new panel, select umh-datasource-v2 from the Data source drop-down menu. It will then fetch the resources from the database. The loading time may depend on your internet speed.

   

   

2. Select the resources in the cascade menu to build your query. DefaultArea and DefaultProductionLine are placeholders for the future implementation of the new data model.

   

   

3. Only the available values for the specified work cell will be fetched from the database. You can then select which data value you want to query.

   

   

4. Next you can specify how to transform the data, depending on what value you selected. For example, all the custom tags will have the aggregation options available. For example if you query a processValue:

   • Time bucket: lets you group data in a time bucket
   • Aggregates: common statistical aggregations (maximum, minimum, sum or count)
   • Handling missing values: lets you choose how missing data should be handled

   

   

Configuration

1. In Grafana, navigate to the Data sources configuration panel.

   

   

2. Select umh-v2-datasource to configure it.

   

   

3. Configurations:

   • Base URL: the URL for the factoryinsight backend. Defaults to http://united-manufacturing-hub-factoryinsight-service/.
   • Enterprise name: previously customerID for the old datasource plugin. Defaults to factoryinsight.
   • API Key: authenticates the API calls to factoryinsight. Can be found with UMHLens by going to Secrets → factoryinsight-secret → apiKey. It should follow the format Basic xxxxxxxx.

   

   
   [i18n] resource_grafanaplugin_baseurlfactoryinsight

2.3.2 - Umh Datasource

We are no longer maintaining this microservice. Use instead our new microservice datasource-v2 for data extraction from factoryinsight.

The umh datasource is a Grafana 8.X compatible plugin, that allows you to fetch resources from a database and build queries for your dashboard.

How it works

1. When creating a new panel, select umh-datasource from the Data source drop-down menu. It will then fetch the resources from the database. The loading time may depend on your internet speed.

   

   

2. Select your query parameters Location, Asset and Value to build your query.

   

   

Configuration

1. In Grafana, navigate to the Data sources configuration panel.

   

   

2. Select umh-datasource to configure it.

   

   

3. Configurations:

   • Base URL: the URL for the factoryinsight backend. Defaults to http://united-manufacturing-hub-factoryinsight-service/.
   • Enterprise name: previously customerID for the old datasource plugin. Defaults to factoryinsight.
   • API Key: authenticates the API calls to factoryinsight. Can be found with UMHLens by going to Secrets → factoryinsight-secret → apiKey. It should follow the format Basic xxxxxxxx.

   

   

2.3.3 - Factoryinput Panel

This plugin is still in development and is not considered stable for production use

Requirements

• A United Manufacturing Hub stack
• External IP or URL to the grafana-proxy
  • In most cases it is the same IP address as your Grafana dashboard.

Getting started

For development, the steps to build the plugin from source are described here.

1. Go to united-manufacturing-hub/grafana-plugins/umh-factoryinput-panel
2. Install dependencies.

yarn install

3. Build plugin in development mode or run in watch mode.

yarn dev

4. Build plugin in production mode (not recommended due to Issue 32336).

yarn build