Components update 8.7 to 8.8
The following sections explain which adjustments must be made to migrate from Camunda 8.7.x to 8.8.x.
Camunda Orchestration Cluster Identity Migration
Camunda 8.8 introduces Orchestration Cluster Identity as a separate component within the Orchestration Cluster, you can learn more about it here.
To ease the effort for migrating data from Management Identity to the new Orchestration Cluster Identity, a Identity Migration Application is provided.
Running the Identity Migration Application
The application can be run locally for manual setups or from a docker container.
Running the migration natively
Using the native distribution of Camunda 8, you can run the Identity Migration Application as follows:
- Mac OS + Linux
- Windows
./bin/identity-migration
./bin/identity-migration.bat
Running the migration using the camunda/camunda:8.8.0
docker image.
You can run the Identity Migration Application by overriding the default entrypoint of the official Camunda 8 Docker image to /usr/local/camunda/bin/identity-migration
, e.g. as such:
docker run --entrypoint /usr/local/camunda/bin/identity-migration camunda/camunda:8.8.0
Monitor completion
You can verify completion of the identity migration by checking for the following log entries:
INFO io.camunda.migration.identity.handler.sm.AuthorizationMigrationHandler - Authorization migration complete: Created {num} out of {num} authorizations. Skipped {num} as they already exist.
INFO io.camunda.migration.identity.handler.sm.ClientMigrationHandler - Client authorization migration complete: Created {num} out of {num} authorizations. Skipped {num} as they already exist.
INFO io.camunda.migration.identity.handler.sm.MappingRuleMigrationHandler - Mapping rule migration completed: Created {num} out of {num} mapping rules, the remaining existed already. Assigned {num} roles out of {num} attempted, the remaining were already assigned. Assigned tenants out of {num} attempted, the remaining were already assigned.
INFO io.camunda.migration.identity.handler.sm.RoleMigrationHandler - Role Migration completed: Created {num} roles out of {num} total roles.
INFO io.camunda.migration.identity.handler.sm.TenantMigrationHandler - Tenant migration completed: Created {num} out of {num} tenants, the remaining existed already.
INFO io.camunda.migration.identity.handler.sm.UserRoleMigrationHandler - Tenant migration completed: User Role membership migration completed: Assigned {num} roles out of {num} roles.
Configuration of the Identity Migration application
The Identity Migration Application supports two scenarios to migrate from, each of those requires different configuration properties and values to be set.
Configuration can generally be set by directly modifying the application.yaml The default location of the config within the Camunda 8 distribution is ./config/application.yaml
and /usr/local/camunda/config/application.yaml
in the docker image respectively.
Alternatively all properties can also be set via Environment variables.
Common configuration for the Identity Migration application
The Identity Migration Application needs to join the Orchestraction Cluster in order to create Identity Access Management records, the values can be taken from equivalent configuration of the Orchestration Cluster nodes.
Furthermore the Identity Migration Application requires full access to the Management Identity API and thus valid client credentials for a client with read
permissions to access it.
- Environment variables
- application.yaml
- Helm Chart
Environment variable | Description | Required value |
---|---|---|
CAMUNDA_MIGRATION_IDENTITY_MODE | Mode of the migration app. | In case of the Camunda Identity 8.7 default (Keycloak) setup: KEYCLOAK . In case of the Camunda Identity 8.7 with OIDC setup: OIDC |
CAMUNDA_MIGRATION_IDENTITY_RESOURCEAUTHORIZATIONSENABLED | Signals whether Resource Authorizations were used in the 8.7, defaults to false . | Must be true in case Resource Authorizations were used. |
CAMUNDA_MIGRATION_IDENTITY_CLUSTER_CLUSTERNAME | Name of the Orchestration cluster. | Same value as used on the cluster for ZEEBE_BROKER_CLUSTER_CLUSTERNAME |
CAMUNDA_MIGRATION_IDENTITY_CLUSTER_CLUSTERSIZE | Size of the Orchestration cluster. | Same value as used on the cluster for ZEEBE_BROKER_CLUSTER_CLUSTERSIZE |
CAMUNDA_MIGRATION_IDENTITY_CLUSTER_PARTITIONSCOUNT | Partition count of the Orchestration cluster. | Same value as used on the cluster for ZEEBE_BROKER_CLUSTER_PARTITIONSCOUNT |
CAMUNDA_MIGRATION_IDENTITY_CLUSTER_REPLICATIONFACTOR | Replication factor of the Orchestration cluster. | Same value as used on the cluster for ZEEBE_BROKER_CLUSTER_REPLICATIONFACTOR |
CAMUNDA_MIGRATION_IDENTITY_CLUSTER_INITIALCONTACTPOINTS | List of known cluster hosts within the Orchestration cluster. | Same value as used on the cluster for ZEEBE_BROKER_CLUSTER_INITIALCONTACTPOINTS |
CAMUNDA_MIGRATION_IDENTITY_MANAGEMENTIDENTITY_AUDIENCE | The audience for accessing the Management Identity API | In case of a default Keycloak setup, it is camunda-identity-resource-server , in case of OIDC, it is user defined. |
CAMUNDA_MIGRATION_IDENTITY_MANAGEMENTIDENTITY_BASEURL | The URL under which Management Identity can be accessed by the Identity Migration Application | Must be a valid absolute URL to the Management Identity server, e.g. http://management-identity:8080 . |
CAMUNDA_MIGRATION_IDENTITY_MANAGEMENTIDENTITY_CLIENTID | The client id for accessing Management Identity. | The clientId set requires to have the read permission granted for the Camunda Identity Resource Server API in Management Identity |
CAMUNDA_MIGRATION_IDENTITY_MANAGEMENTIDENTITY_CLIENTSECRET | The client secret for accessing Management Identity. | Matching secret for configured clientId. |
CAMUNDA_MIGRATION_IDENTITY_MANAGEMENTIDENTITY_ISSUERBACKENDURL | The URL under which the Token Issuer can be accessed. | Use the same as used by Management Identity for CAMUNDA_IDENTITY_ISSUER_BACKEND_URL . |
Application.yaml property | Description | Required value |
---|---|---|
camunda.migration.identity.mode | Mode of the migration app. | In case of the Camunda Identity 8.7 default (Keycloak) setup: KEYCLOAK . In case of the Camunda Identity 8.7 with OIDC setup: OIDC |
camunda.migration.identity.resource-authorizations-enabled | Signals whether Resource Authorizations were used in the 8.7, defaults to false . | Must be true in case Resource Authorizations were used. |
camunda.migration.identity.cluster.cluster-name | Name of the Orchestration cluster. | Same value as used on the cluster for zeebe.broker.cluster.cluster-name |
camunda.migration.identity.cluster.cluster-size | Size of the Orchestration cluster. | Same value as used on the cluster for zeebe.broker.cluster.cluster-size |
camunda.migration.identity.cluster.partitions-count | Partition count of the Orchestration cluster. | Same value as used on the cluster for zeebe.broker.cluster.partitions-count |
camunda.migration.identity.cluster.replication-factor | Replication factor of the Orchestration cluster. | Same value as used on the cluster for zeebe.broker.cluster.replication-factor |
camunda.migration.identity.cluster.initial-contact-points | List of known cluster hosts within the Orchestration cluster. | Same value as used on the cluster for zeebe.broker.cluster.initial-contact-points |
camunda.migration.identity.management-identity.audience | The audience for accessing the Management Identity API | In case of a default Keycloak setup, it is camunda-identity-resource-server , in case of OIDC, it is user defined. |
camunda.migration.identity.management-identity.base-url | The URL under which Management Identity can be accessed by the Identity Migration Application | Must be a valid absolute URL to the Management Identity server, e.g. http://management-identity:8080 . |
camunda.migration.identity.management-identity.client-id | The client id for accessing Management Identity. | The clientId set requires to have the read permission granted for the Camunda Identity Resource Server API in Management Identity |
camunda.migration.identity.management-identity.client-secret | The client secret for accessing Management Identity. | Matching secret for configured clientId. |
camunda.migration.identity.management-identity.issuer-backend-url | The URL under which the Token Issuer can be accessed. | Use the same as used by Management Identity for CAMUNDA_IDENTITY_ISSUER_BACKEND_URL . |
In the Helm chart deployments, the Identity Migration app runs as a Kubernetes job that operates independently from other migration processes. Key characteristics include:
- Idempotent operation: Safe to rerun multiple times without creating duplicates
- Standalone execution: Runs separately from the data migration process
- Clear error reporting: Provides detailed logs for troubleshooting
- One-time setup: Once successful, you can safely clean up the migration resources
Migration client setup
The migration requires a dedicated Keycloak client with read-only access to your Identity Resource Server.
Automatic configuration (recommended):
When you enable the Identity migration, the helm charts automatically configure the required migration
client for you.
Manual configuration (when needed):
If you manually configure Keycloak clients in your values.yaml
, you must also define the migration client:
identity:
env:
- name: KEYCLOAK_CLIENTS_2_ID
value: migration
- name: KEYCLOAK_CLIENTS_2_NAME
value: Migration
- name: KEYCLOAK_CLIENTS_2_TYPE
value: CONFIDENTIAL
- name: KEYCLOAK_CLIENTS_2_PERMISSIONS_0_RESOURCE_SERVER_ID
value: camunda-identity-resource-server
- name: KEYCLOAK_CLIENTS_2_PERMISSIONS_0_DEFINITION
value: read
- name: KEYCLOAK_CLIENTS_2_SECRET
valueFrom:
secretKeyRef:
name: camunda-credentials
key: identity-client-secret
Adjust the client index number (2
in this example) based on your existing client configuration. Each client requires a unique index.
Manual migration instructions
Roles & Application Permissions
Permissions formerly assigned to Management Identity Roles and Management Identity Applications require the following equivalent Orchestration Cluster Identity Authorizations setup.
You need to manually recreate the Roles you need and grant them the equivalent Authorizations for the former Management Identity Permission as follows:
Management Identity Permission | Equivalent Camunda Orchestration Cluster 8.8 Authorizations | |||
---|---|---|---|---|
Component | Permission | Resource Type | Resource ID | Permission(s) |
Operate | read | BATCH | * | READ |
COMPONENT | Operate | ACCESS | ||
MESSAGE | * | READ | ||
PROCESS_DEFINITION | * | READ_PROCESS_DEFINITION , READ_PROCESS_INSTANCE | ||
DECISION_DEFINITION | * | READ_DECISION_DEFINITION , READ_DECISION_INSTANCE | ||
DECISION_REQUIREMENTS_DEFINITION | * | READ | ||
write | BATCH | * | CREATE , READ , UPDATE | |
COMPONENT | Operate | ACCESS | ||
RESOURCE | * | DELETE_FORM , DELETE_PROCESS , DELETE_DRD , DELETE_RESOURCE | ||
PROCESS_DEFINITION | * | READ_PROCESS_DEFINITION , READ_PROCESS_INSTANCE , DELETE_PROCESS_INSTANCE , UPDATE_PROCESS_INSTANCE , MODIFY_PROCESS_INSTANCE CANCEL_PROCESS_INSTANCE | ||
DECISION_DEFINITION | * | READ_DECISION_DEFINITION , READ_DECISION_INSTANCE , CREATE_DECISION_INSTANCE , DELETE_DECISION_INSTANCE | ||
DECISION_REQUIREMENTS_DEFINITION | * | READ , UPDATE , DELETE | ||
Tasklist | read | COMPONENT | Tasklist | ACCESS |
PROCESS_DEFINITION | * | READ_PROCESS_DEFINITION , READ_USER_TASK | ||
write | COMPONENT | Tasklist | ACCESS | |
PROCESS_DEFINITION | * | READ_PROCESS_DEFINITION , READ_USER_TASK , UPDATE_USER_TASK | ||
Identity | read | AUTHORIZATIONS | * | READ |
COMPONENT | Identity | ACCESS | ||
GROUP | * | READ | ||
ROLES | * | READ | ||
TENANTS | * | READ | ||
USERS | * | READ | ||
read:users | COMPONENT | Identity | ACCESS | |
USERS | * | READ | ||
ROLES | * | READ | ||
write | AUTHORIZATIONS | * | CREATE , READ , UPDATE , DELETE | |
COMPONENT | Identity | ACCESS | ||
GROUP | * | CREATE , READ , UPDATE , DELETE | ||
ROLES | * | CREATE , READ , UPDATE , DELETE | ||
TENANTS | * | CREATE , READ , UPDATE , DELETE | ||
USERS | * | READ |
User Role membership
You can recreate Management Identity Role memberships using the Orchestration Cluster Identity Role Management.
Groups
You can recreate Groups setup in the Management Identity Groups using the Orchestration Cluster Identity Group Management.
Alernatively you can make use of the Bring your own Groups feature and just recreate the Orchestration Cluster Authorization Records granting permissions to those groups. This is what the Identity Migration Application defaults to.
Resource Authorizations
The former Management Identity Resource Authorizations are replaced with a more fine grained Orchestration Cluster Authorization Management. The following table lists the equivalent authorization records to create, to represent for a former Management Identity Resource Authorizations.
Management Identity Resource Authorizations | Equivalent Camunda Orchestration Cluster 8.8 Authorizations | ||
---|---|---|---|
Resource Type | Permission | Resource Type | Permission(s) |
PROCESS | Read | PROCESS_DEFINITION | READ_PROCESS_DEFINITION , READ_PROCESS_INSTANCE |
PROCESS | Delete | RESOURCE | DELETE_PROCESS |
PROCESS | Update process instance | PROCESS_DEFINITION | UPDATE_PROCESS_INSTANCE |
PROCESS | Delete process instance | PROCESS_DEFINITION | DELETE_PROCESS_INSTANCE |
PROCESS | Start process instance | PROCESS_DEFINITION | CREATE_PROCESS_INSTANCE |
DECISION | Read | DECISION_DEFINITION | READ_DECISION_DEFINITION , READ_DECISION_INSTANCE |
DECISION | Delete | RESOURCE | DELETE_DRD |
Tenants
You can recreate Management Identity Tenants using the Orchestration Cluster Identity Tenant Management.
Mapping Rules
You can recreate Management Identity Mapping Rules using the Orchestration Cluster Identity Mapping Rule Management.
Camunda Exporter and harmonized data model
The following Camunda 8.7 to 8.8 update guide covers changes related to the new harmonized data model, and the newly introduced Camunda Exporter. For additional details about the introduced exporter and related changes, see the 8.8 announcements.
With the Camunda Exporter introduction, Camunda is compatible with Elasticsearch 8.16+ and no longer supports older Elasticsearch versions. See supported environments.
Ensure the exporter/import backlog is small
For data from 8.7 to be imported, the importers must run first and complete before the Camunda Exporter can start.
The importer reads data from existing Zeebe Elasticsearch indices, and when it detects data from newer versions, it marks itself as complete. The Camunda Exporter then starts exporting only when the importer is done.
To reduce the potential of building a backlog, the user needs to make sure to drain the backlog queue as much as possible before the update.
Otherwise, the backlog can increase and cause web application data to become out of date. Zeebe will be not able to clean up data, potentially running out of disc space. This will produce downtime.
The backlog can be validated via the following Prometheus metrics:
operate_import_time_seconds_sum
operate_import_time_seconds_count
For example, the following query shows the import latency. If the result is in a range of seconds (ideally below 10 seconds), it is fine to update.
sum(rate(operate_import_time_seconds_sum{namespace=~"$namespace", partition=~"$partition", pod=~"$pod"}[$__rate_interval]))
/ sum(rate(operate_import_time_seconds_count{namespace=~"$namespace", partition=~"$partition", pod=~"$pod"}[$__rate_interval]))
Data migration
Camunda 8.8 provides dedicated migration applications to ensure data integrity and compatibility during upgrades. Upgrades must be performed sequentially through each minor version (for example, 8.6 → 8.7 → 8.8), as skipping versions is not supported.
The migration applications are located in the camunda/bin/
directory. You can run them locally, targeting the same Elasticsearch or OpenSearch instance, either as standalone tools or alongside the rest of the Camunda 8 application. You can also use the data-migration
binary which wraps all the migrations in a single application (excluding identity-migration
).
Data migration requires a configured secondary storage. All migrations share the same base configuration but can be individually adjusted as needed. Prometheus metrics for migration progress are exposed at the :9600/actuator/prometheus
endpoint.
For details about each migration, see the sections below.
- Process Migration
- Task Migration
- Metrics Migration
Scope
The process migration component unifies the previously separate operate-process
and tasklist-process
indices. To achieve this, it extracts the Camunda Form–related data from deployments that was previously used only by Tasklist.
Depends on the Operate importer being completed.
Configuration
Spring profile: process-migration
camunda:
migration:
process:
# How many process definitions are processed per round
# Default 20
batchSize: 5
# The timeout/count down after importer has finished before the migration app should stop
# Default PT1M
importerFinishedTimeout: PT1M
# The timeout for the migration
# Default PT2H
timeout: PT2H
# Retry properties, exponential backoff
retry:
# How many times do we retry failing operations until we stop the migration process
maxRetries: 5
# The minimum retry delay applied to backoff
# Default PT1S
minRetryDelay: PT10S
# The maximum retry delay applied to backoff
# Default PT1M
maxRetryDelay: PT1M
Exposed metrics
camunda_migration_processes_migrated
- Number of migrated processes so farcamunda_migration_processes_rounds
- Number of migration rounds performed so farcamunda_migration_processes_round_time
- Time taken to complete one migration roundcamunda_migration_processes_single_process_time
- Time taken to process a single process definition
Scope
This migration component updates the tasklist-task
runtime and dated indices to remove the dynamic:true
requirement from the tasklist-task.customHeaders
field mapping.
This migration also upgrades all the tasklist-task
indices to 8.8.0. The original runtime tasklist-task
index (8.5.0) is retained for data safety. If cluster retention is enabled, an ILM or ISM policy is applied to this original runtime index (8.5.0) for the configured retention period (default: 30 days).
Depends on theTasklist importer being completed.
⚠️ Side effects
Because this migration actively moves data from indices, some temporary side effects may occur in the Orchestration Cluster:
- Some tasks may not be visible in the Tasklist UI or returned by the Tasklist API.
- With Tasklist API v1, some tasks cannot be assigned, unassigned, or completed. These requests return
404 Not Found
. - With Tasklist API v2, tasks can be assigned, unassigned, or completed, but attempts to retrieve them (for example, via search) may return
500 Internal Server Error
. - Refreshing the Tasklist UI while a task is selected may return
404 Not Found
if the task is still being migrated. - Refreshing the Tasklist UI while a task is selected may return
500 Internal Server Error
if the task was completed or reassigned via API v2 but not yet fully migrated. - Archiving is disabled during this period.
These issues resolve automatically once migration is complete.
Configuration
Spring profile: task-migration
camunda:
migration:
tasks:
# How many tasks are processed per round
# Default 20
batchSize: 20
# Retention age for the old `tasklist-task-8.5.0` runtime index. Applies only if global retention is enabled
# Default 30d
legacyIndexRetentionAge: 30d
# Retry properties, exponential backoff
retry:
maxRetries: 5
minRetryDelay: PT10S
maxRetryDelay: PT1M
Exposed metrics
camunda_migration_tasks_migrated
- Number of tasks processed so farcamunda_migration_tasks_rounds
- Total number of task batch update roundscamunda_migration_tasks_round_time
- Duration of a task batch migrationcamunda_migration_tasks_single_task_time
- Duration to process a single task
Scope
This migration component transfers existing usage metrics from Operate and Tasklist into the new data structure of their respective indices.
It consists of two parts: Operate metrics and Tasklist usage metrics, referenced as metrics
and tu-metrics
in the configuration.
Each migration depends on it's corresponding importer being completed.
Configuration
Spring profile: usage-metric-migration
camunda:
migration:
metrics:
timeout: PT2H
retry:
maxRetries: 5
minRetryDelay: PT10S
maxRetryDelay: PT1M
tu-metrics:
timeout: PT2H
retry:
maxRetries: 5
minRetryDelay: PT10S
maxRetryDelay: PT1M
Exposed metrics
camunda_migration_operate_reindex_time
- Duration to reindex the Operate metrics indexcamunda_migration_operate_task_importer_finished
- Duration until the Operate importer finishescamunda_migration_tasklist_reindex_time
- Duration to reindex the Tasklist metrics indexcamunda_migration_tasklist_task_importer_finished
- Duration until the Tasklist importer finishes
Standalone Migration (locally)
When running the application separately (locally for example), the configuration must be made public to the migration application.
For example, the location of the application.yaml
can be provided for local execution:
SPRING_CONFIG_ADDITIONALLOCATION=/path/to/application.yaml ./camunda/bin/process-migration
Integration in the standalone Camunda application
When running the standalone Camunda application, any migration application can be executed within it. You can enable it using the respective Spring profile.
Monitor completion
Apart from the respective exposed Prometheus metrics, completion of the data migration can also be verified from the log output. Example:
INFO io.camunda.migration.task.TaskMigrator - Task Migration completed successfully
INFO io.camunda.migration.process.ProcessMigrator - Process Migration completed
INFO io.camunda.migration.usagemetric.OperateMetricMigrator - Reindex task {id} completed successfully
INFO io.camunda.migration.usagemetric.TasklistMetricMigrator - Reindex task {id} completed successfully
Turn off importers after completion
Importers are only necessary until the migration is successfully completed. Then, they can be safely turned off.
You can enable the importers by setting:
camunda.operate.importerEnabled: true
camunda.tasklist.importerEnabled: true
To detect whether importers are done, users can validate the state in the Elasticsearch/OpenSearch index:
tasklist-import-position-8.2.0_
operate-import-position-8.3.0_
If all entries show the completed
property set to true
, the importing is done.
Alternatively, metrics can be validated via:
operate_import_completed
tasklist_import_completed
These properties can be used in the following example query:
sum(tasklist_import_completed{namespace="$namespace"}) by (importPositionAlias)
Update any custom prefixes
The new harmonized index schema introduces one common index prefix. To update to Camunda 8.8, existing indices using custom prefixes must be migrated to a common prefix, so the Camunda Exporter, REST API, and other consumers can be configured correctly.
Prefix migration
This step is required if your installation consists of custom prefixed indices.
Prefix migration modifies the secondary storage in preparation for the Camunda 8.8 update. These changes are irreversible, so creating a cluster backup is required. To ensure data integrity, prefix migration must be performed during downtime. All data in the secondary storage must be migrated to a new prefix, and no access to Camunda indices should occur during this window.
Do not restart the cluster on the previous Camunda 8.7 version.
Complete the following steps to migrate any custom prefixes:
- Shut down your Camunda 8.7 cluster (for example, reduce the replicas to zero). No traffic can be executed against your to-be-migrated ES/OS prefixes.
- In the
camunda/bin
directory, theprefix-migration
script will execute the prefix migration, which can be executed locally.- Make sure that your configuration in
camunda/config
has set the previous custom prefixes, and your configuration is made public to the script (for example, usingSPRING_CONFIG_ADDITIONALLOCATION=/path/to/application.yaml ./camunda/bin/prefix-migration
). - Alternatively, you can set the configuration via environment variables
- Make sure that your configuration in
Example execution of the prefix migration:
- Elasticsearch
- OpenSearch
Environment variables configuration:
export CAMUNDA_OPERATE_ELASTICSEARCH_INDEX_PREFIX=old-operate-prefix ;\
export CAMUNDA_TASKLIST_ELASTICSEARCH_INDEX_PREFIX=old-tasklist-prefix ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_ELASTICSEARCH_INDEXPREFIX=some-new-prefix ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_ELASTICSEARCH_URL=http://localhost:9200 ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_TYPE=elasticsearch ;\
./bin/prefix-migration
YAML file configuration:
camunda:
tasklist:
elasticsearch-index-prefix: old-tasklist-prefix
operate:
elasticsearch-index-prefix: old-operate-prefix
data:
secondary-storage:
type: elasticsearch
elasticsearch:
url: http://localhost:9200
index-prefix: some-new-prefix
Environment variables configuration:
export CAMUNDA_OPERATE_OPENSEARCH_INDEX_PREFIX=old-operate-prefix ;\
export CAMUNDA_TASKLIST_OPENSEARCH_INDEX_PREFIX=old-tasklist-prefix ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_OPENSEARCH_INDEXPREFIX=some-new-prefix ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_OPENSEARCH_URL=http://localhost:9200 ;\
export CAMUNDA_DATA_SECONDARYSTORAGE_TYPE=opensearch ;\
./bin/prefix-migration
YAML file configuration:
camunda:
tasklist:
opensearch-index-prefix: old-tasklist-prefix
operate:
opensearch-index-prefix: old-operate-prefix
data:
secondary-storage:
type: opensearch
opensearch:
url: http://localhost:9200
index-prefix: some-new-prefix
If a custom prefix is supplied for only one of Tasklist or Operate, set it on the respective index-prefix
parameter.
After the prefix migration completes, your secondary storage is ready to proceed with the update to Camunda 8.8.
Once the migration is finished, apply the following configuration changes before updating to version 8.8.
Given a new desired prefix of some-new-prefix
:
CAMUNDA_DATA_SECONDARY-STORAGE_ELASTICSEARCH_INDEXPREFIX=some-new-prefix
ZEEBE_BROKER_EXPORTERS_CAMUNDA_ARGS_CONNECT_INDEXPREFIX=some-new-prefix
Single Elasticsearch/OpenSearch instance
Using more than one isolated Elasticsearch/OpenSearch instance for exported Zeebe, Operate, and Tasklist data is no longer supported. If your environment uses multiple Elasticsearch/OpenSearch instances, you must manually migrate the data from each to a single Elasticsearch/OpenSearch cluster before updating to Camunda 8.8. The migration should target Zeebe, Operate, and Tasklist indices, index templates, aliases, and ILM policies.
This step must be performed before all other migration steps.
Orchestration Cluster REST API
Removed deprecated OpenAPI objects
With the Camunda 8.8 release, deprecated API objects containing number keys have been removed, including the
corresponding application/vnd.camunda.api.keys.number+json
content type header.
In previous releases, entity keys were transitioned from integer (int64)
to string
types, and deprecated
integer (int64)
keys were still supported. As of the 8.8 release, support for integer (int64)
keys has been removed.
To update to Camunda 8.8, API objects using integer (int64)
keys must be updated to use string
keys and the
application/json
header.
For more information about the key attribute type change, see the 8.7 API key attributes overview.
Streamlined variable OpenAPI objects
The OpenAPI specification streamlines the objects used for variable filtering in search requests. The supported Camunda Java client handles those changes transparently. The affected API endpoints are alpha feature endpoints and thus already marked as subject to change in 8.6 and 8.7.
If you are generating a custom REST client using the specification, consider the following changes:
- The unified
VariableValueFilterProperty
replaces theProcessInstanceVariableFilterRequest
andUserTaskVariableFilterRequest
in the OpenAPI spec. - For naming consistency, the
UserTaskVariableFilter
replaces theVariableUserTaskFilterRequest
.
Web application permission migration
With the changes to the authorization system, access control for public APIs and component UIs has been updated.
As a result, existing user and client permissions must be adjusted to continue using deprecated component API endpoints, as well as to retain visibility of information in applications and perform operations.
Migrate Operate V1 endpoint permissions
A detailed guide for migrating permissions for the V1 API endpoints can be found in the public API migration guides.
Migrate Operate and Tasklist access permissions
The new permissions define what is displayed to users and which actions can be performed in the component UIs. They provide more fine-grained control, for example, granting a user read access to only a specific process.
To maintain the same access, views and control over modifications in the Operate and Tasklist UI as in previous versions, either run the Identity migration application or refer to the manual migration instructions.
The authorization concept for batch operations in Operate has changed from user-based permissions to the new resource permissions. After the upgrade users will not be able to see batch operations anymore (including their own) unless they were explicitly given read permissions on all batch operations.
To fully leverage the new authorizations and customize access, refer to the access control documentation for Operate and Tasklist.
Exported records
USER_TASK
records
To support user task listeners, some backward incompatible changes were necessary to the exported USER_TASK
records.
assignee
no longer provided in CREATED
event
Previously, when a user task was activating with a specified assignee
,
we appended the following events of the USER_TASK
value type:
CREATING
withassignee
property as providedCREATED
withassignee
property as provided
The ASSIGNING
and ASSIGNED
events were not appended in this case.
To support the new User Task Listeners feature, the assignee
value will not be filled in the CREATED
event anymore.
With 8.8, the following events are now appended:
CREATING
withassignee
property as providedCREATED
withassignee
always""
(empty string)ASSIGNING
withassignee
property as providedASSIGNED
withassignee
property as provided
ASSIGNING
has become CLAIMING
for CLAIM
operation
When claiming a user task, we previously appended the following records of the USER_TASK
value type:
CLAIM
ASSIGNING
ASSIGNED
A new CLAIMING
intent was introduced to distinguish between claiming and regular assigning.
We now append the following records when claiming a user task:
CLAIM
CLAIMING
ASSIGNED
The ASSIGNING
event is still appended for assigning a user task.
In that case, we append the following records:
ASSIGN
ASSIGNING
ASSIGNED
Zeebe record types
With the introduction of the Camunda Exporter, the Elasticsearch and OpenSearch Exporter no longer export all record types by default. Therefore, not all zeebe-record
indices will be populated.
This only affects records exported with the 8.8 version. During the migration, leftover 8.7 records are exported as normal.
The record types that continue to be exported by default are the following:
DEPLOYMENT
PROCESS
PROCESS_INSTANCE
VARIABLE
USER_TASK
INCIDENT
JOB
To export other record types, review includeEnabledRecords
for Elasticsearch or OpenSearch exporter configuration.
Exporter deletion
Exporter configuration removal no longer deletes the exporter from the system. Instead, the exporter enters a blocked state to safeguard against data loss caused by accidental or manual configuration changes. To permanently remove an exporter, use the Exporters API.
Connectors
Email connector
Starting with the 8.8 release, angle brackets (<
and >
) are no longer removed from the messageId
.
Logging
Default configuration
The default logging configuration is included inline for quick reference.
See the full logging documentation for advanced settings and details.
Rolling file appender
The RollingFile
appender is disabled by default.
To enable it, set the appropriate environment variable for each component:
ZEEBE_LOG_APPENDER=RollingFile
Other supported component variables:
OPERATE_LOG_APPENDER
IDENTITY_LOG_APPENDER
TASKLIST_LOG_APPENDER
Make sure to use the correct variable for the component you are configuring.
Pattern layout changes
The default log pattern has been updated:
Old pattern:
[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%t] %notEmpty{[%X] }%-5level%n\t%logger{36} - %msg%n
New pattern:
%d{HH:mm:ss.SSS} [%t] %notEmpty{[%X] }%-5level%logger{1.1.1.*} - %msg%n
Feature | Old pattern | New pattern |
---|---|---|
Timestamp | Full date and time | Time only |
Logger name | Up to 36 characters | Package initials + full class name |
Newline after level | Yes | No |
Tab before logger | Yes | No |
Elasticsearch
Starting in 8.8, the default replica count for Camunda indices in Elasticsearch and OpenSearch changes from 0 to 1. This ensures that if a node goes down, Camunda is not blocked by a temporary outage of the secondary data store.
If you already override this value, no action is required. Otherwise, review the guidance below to adjust capacity or revert the change.
Throughout this section, references to Elasticsearch (ES) also apply to OpenSearch (OS). For simplicity, ES is used as the default term.
Single-node cluster
If the current ES cluster has only one node, setting the replica value to 1 will cause the cluster state to go yellow.
This means all primary shards are allocated, but some replicas are unassigned.
To resolve this issue:
-
Increase the number of master-eligible nodes to at least 2.
-
Refer to the Bitnami Elasticsearch Helm chart values, specifically the
master.replicaCount
field.⚠️ Important: Bitnami recommends scaling the ES cluster down first for consistency reasons. Follow their guidance carefully.
If you install Elasticsearch using the Camunda Helm chart, the default master.replicaCount
is 3
, resulting in a three-node cluster.
For other setups, you can apply the same logic by setting elasticsearch.master.replicaCount
.
Once complete, continue with Multi-node cluster.
Multi-node cluster
Starting in 8.8, the default replica count for indices is 1. If your cluster previously used a replica count of 0, this will roughly double disk usage.
To ensure the ES cluster can reach a green status, make sure the disk capacity is at least 2.5× the previously used storage.
This calculation accounts for watermarks, overhead, and growth.
For example:
- In 8.7, node disk usage is 30/50 GB, 35/50 GB, and 40/50 GB.
- The most used node has 40 GB.
- To ensure replicas fit on other nodes, resize all disks to at least 100 GB (40 × 2.5).
Resize Elasticsearch disk
If the current disk size is too small for replicas, assign more disk space. Whether a Persistent Volume Claim (PVC) is expandable depends on the underlying StorageClass
.
For information on expanding PVCs using the calculations above, see:
For managed services, see:
- Elastic Cloud: Fix master nodes out of disk
- AWS OpenSearch: Modifying an EBS Volume
(relevant because AWS OpenSearch is backed by EBS volumes)
Roll back
If you encounter issues after the upgrade and replica increase, revert the replica count to 0 in the Zeebe deployment and restart.
Zeebe pods will trigger the processes which update the indices back to 0 replicas.
Set one of the following:
- YAML:
camunda.database.index.numberOfReplicas: 0
- Env var:
CAMUNDA_DATABASE_INDEX_NUMBER_OF_REPLICAS=0
Web Modeler
Cluster configuration
Removed authentication methods
The previously deprecated authentication methods OAUTH
and CLIENT_CREDENTIALS
are no longer supported.
If you still use one of them, update the cluster configuration in your Web Modeler installation:
- Replace
OAUTH
withBEARER_TOKEN
(the value was just renamed). - Instead of
CLIENT_CREDENTIALS
you can useBEARER_TOKEN
authentication if the cluster is also on version 8.8 and uses OIDC authentication with the same identity provider as Web Modeler. The previous limitation that the identity provider must support access tokens with multiple audiences no longer applies. However, you need to ensure that the cluster accepts Web Modeler's token audience by including it in the configured list of audiences.
Changed configuration options
The available configuration options now depend on the version of the cluster. For version 8.8, new configuration options are required. If your existing configuration contains a cluster that was updated to version 8.8 from a previous version, you need to modify the configuration:
- Change
CAMUNDA_MODELER_CLUSTERS_*_URL_ZEEBE_GRPC
toCAMUNDA_MODELER_CLUSTERS_*_URL_GRPC
. - Change
CAMUNDA_MODELER_CLUSTERS_*_URL_ZEEBE_REST
toCAMUNDA_MODELER_CLUSTERS_*_URL_REST
. - Add
CAMUNDA_MODELER_CLUSTERS_*_URL_WEBAPP
. - Add
CAMUNDA_MODELER_CLUSTERS_*_AUTHORIZATIONS_ENABLED
. - Remove
CAMUNDA_MODELER_CLUSTERS_*_URL_OPERATE
. - Remove
CAMUNDA_MODELER_CLUSTERS_*_URL_TASKLIST
.