Upgrade Guidelines¶

Upgrading DSW¶

Warning

Backup database and other important data (e.g., configuration) before upgrade!

Using Docker¶

In case of using Docker, just use the tag in docker-compose.yml or pull the new Docker image and restart using down/up:

$ docker pull datastewardshipwizard/wizard-server
$ docker pull datastewardshipwizard/wizard-client
$ docker pull datastewardshipwizard/document-worker
$ docker pull datastewardshipwizard/mailer
$ docker-compose down
$ docker-compose up -d

Other setup (using Git)¶

All we need to do is download or checkout the new version from GitHub repositories and rebuild the application (according to the guidelines above):

$ git checkout vX.Y.Z

Upgrade process¶

Usually, nothing special is required for the upgrade. Internal structure changes are migrated automatically using DB migrations and Metamodel migrations (since 1.8.0). See below the changes that need to be done by us (since 1.10.0):

Warning

Make sure to stop document-worker and mailer before upgrading to the next version. Run wizard-server first, then run the workers. Otherwise the database migrations might not work correctly.

4.29.X to 4.30.X¶

(nothing)

4.28.X to 4.29.X¶

(nothing)

4.27.X to 4.28.X¶

Project Importers were removed. They can be replaced by Plugins.
API endpoints in sections related to Document Templates and Knowledge Models were updated. The primary identifier was changed from ID to UUID. Please check the API documentation for the new endpoints.
Due to Document Template changes, it is necessary to migrate Document Templates in S3. You can use this script: https://github.com/ds-wizard/dsw-templates-migration-script

4.26.X to 4.27.X¶

API endpoints in the sections related to Locales were updated. The primary identifier was changed from ID to UUID. Please check the API documentation for the new endpoints.
Due to Locale changes, it is necessary to migrate Locales in S3. You can use this script: https://github.com/ds-wizard/dsw-locale-migration-script

4.25.X to 4.26.X¶

API Endpoints in the sections related to Questionnaire were renamed to Project to match the DSW UI. Please check the API documentation for the new endpoints.
Due to changes in the API, it is necessary to rename questionnaire-files in S3 to project-files.
Document Template metamodel version is raised (from 17.0 to 17.1) due to new type of API Integration in KM and way of versioning document template metamodel, for details check Document Template Metamodel Versions.

4.24.X to 4.25.X¶

API Endpoints in the sections related to Knowledge Model and Knowledge Model Editor have been updated. Please check the API documentation for the new endpoints.
All Document Submissions with no Submission service defined will be deleted during migration. This can happen if the Submission service used by the Document Submission was deleted or renamed.
It is necessary to finish all Knowledge Model migrations, either by completing or deleting them, before upgrading. The upgrade will delete any ongoing migrations.

4.23.X to 4.24.X¶

(nothing)

4.22.X to 4.23.X¶

(nothing)

4.21.X to 4.22.X¶

Document template metamodel version is raised (from 16 to 17.0) due to new type of API Integration in KM and way of versioning document template metamodel, for details check Document Template Metamodel Versions.
It is necessary to delete any user properties belonging to Document Submission that no longer exist in the system. Otherwise, the migration will fail. This can be done either through the UI or directly in the database.
It is recommended to migrate API integrations to the new version as soon as possible. The legacy API integration will be removed in the future.

4.20.X to 4.21.X¶

Before updating check following, otherwise the migration will fail:

All authentication services have unique IDs
All submission services have unique IDs
Document templates and their versions used by submission services are present in the instance

4.19.X to 4.20.X¶

(nothing)

4.18.X to 4.19.X¶

(nothing)

4.17.X to 4.18.X¶

Updating will remove all existing locales. They need to be backed up and re-imported after the update.

4.16.X to 4.17.X¶

(nothing)

4.15.X to 4.16.X¶

(nothing)

4.14.X to 4.15.X¶

(nothing)

4.13.X to 4.14.X¶

(nothing)

4.12.X to 4.13.X¶

Document template metamodel version is raised (from 15 to 16) as Value Question has now added Validations (no breaking changes), for details check Document Template Metamodel Versions.

4.11.X to 4.12.X¶

Document template metamodel version is raised (from 14 to 15) as File Question has been added (no breaking changes), for details check Document Template Metamodel Versions.

4.10.X to 4.11.X¶

(nothing)

4.9.X to 4.10.X¶

Document template metamodel version is raised (from 13 to 14) and requires updates in document templates, for details check Document Template Metamodel Versions.

4.8.X to 4.9.X¶

(nothing)

4.7.X to 4.8.X¶

(nothing)

4.6.X to 4.7.X¶

(nothing)

4.5.X to 4.6.X¶

(nothing)

4.4.X to 4.5.X¶

(nothing)

4.3.X to 4.4.X¶

(nothing)

4.2.X to 4.3.X¶

The document template metamodel version is raised to 13.

4.1.X to 4.2.X¶

(nothing)

4.0.X to 4.1.X¶

(breaking) DSW Integration Widget SDK and DSW Importer SDK for importers are now deprecated. DSW Integration SDK should be used instead.
The document template metamodel version is raised to 12.

3.28.X to 4.0.X¶

(breaking) The client runs on the nested route /wizard, and the server runs on the nested route /wizard-api. These changes must be reflected in the deployment configuration (such as routing in a reverse proxy) and the clientUrl in the Server Configuration.
(breaking) Client Configuration for a custom logo and theme no longer uses SASS and needs to be updated accordingly.
(breaking) API Keys created in previous versions will no longer work.
(breaking) The callback URL for OpenID (such as Google or Life Science Login) has changed due to the nested route, so it has to be set up, for example:
- Before: https://researchers.ds-wizard.org/auth/google/callback
- After: https://researchers.ds-wizard.org/wizard/auth/google/callback

3.27.X to 3.28.X¶

(nothing)

3.26.X to 3.27.X¶

(nothing)

3.25.X to 3.26.X¶

(nothing)

3.24.X to 3.25.X¶

(nothing)

3.23.X to 3.24.X¶

(nothing)

3.22.X to 3.23.X¶

(breaking) The JWT signing has been changed to RS256 and you need to add rsaPrivateKey in configuration file (see Server Configuration).
(breaking) The location of configuration files has been changed and unified across components, check the deployment example for details. The main configuration file is located in /app/config/application.yml path which can be adjusted APPLICATION_CONFIG_PATH.

3.21.X to 3.22.X¶

(nothing)

3.20.X to 3.21.X¶

(breaking) The wizard-client container now exposes a different port (as all images are now root-less): 8080 instance of 80.
(breaking) The S3 service must be now publicly available, thus the S3 URL configured via Server Configuration must be reachable by users to support download of documents or document preview.

3.19.X to 3.20.X¶

The document template metamodel version is raised to 11. All templates must be updated (changes are only minor in template.json files, see Template Development section for more information).

3.18.X to 3.19.X¶

(nothing)

3.17.X to 3.18.X¶

(nothing)

3.16.X to 3.17.X¶

If we are upgrading from the older version then 3.16.X we need to first upgrade to version 3.16.X.

3.15.X to 3.16.X¶

(nothing)

3.14.X to 3.15.X¶

(nothing)

3.13.X to 3.14.1¶

(nothing)

3.13.X to 3.14.X¶

(nothing)

3.12.X to 3.13.X¶

(nothing)

3.11.X to 3.12.X¶

(nothing)

3.10.X to 3.11.X¶

(optional) We can now use integration.yaml configuration in Settings instead of the file store on FS and mounted to the Docker container.

3.9.X to 3.10.X¶

Standalone mailer component has been introduced. We need to adjust our deployment (e.g., docker-compose.yml) accordingly (see deployment-example).

3.8.X to 3.9.X¶

(nothing)

3.7.X to 3.8.X¶

All KM migrations must be finished (completed or deleted); otherwise, the upgrade of the backend (database) will fail with the corresponding message in the logs.

3.6.X to 3.7.X¶

(nothing)

3.5.X to 3.6.X¶

(nothing)

3.4.X to 3.5.X¶

The template metamodel version has been updated (to v5). Updating all document templates is needed (annotations were added, so we can safely change version 4 to version 5 without breaking anything).
All KM migrations must be finished (completed or deleted); otherwise, the upgrade of the backend (database) will fail with the corresponding message in the logs.

3.3.X to 3.4.X¶

(nothing)

3.2.X to 3.3.X¶

(nothing)

3.1.X to 3.2.X¶

The template metamodel version has been updated (to v4). Updating all document templates is needed.
All knowledge models have (after the automatic data migration) the default metrics and phases that can be changed in KM Editor.

3.0.X to 3.1.X¶

As an administrator, we should either disable the “Project Templates” feature (Settings - Projects - Project Creation, select “Custom only”) or prepare some project templates for our users to avoid confusion.

2.14.X to 3.0.X¶

All data must be migrated as we switched from MongoDB and RabbitMQ to PostgreSQL and S3. To support data migration, we provide dsw2to3 tool with step-by-step instructions.

2.13.X to 2.14.X¶

(nothing)

2.12.X to 2.13.X¶

(nothing)

2.11.X to 2.12.X¶

The metamodel for templates has been upgraded, and accessing the reply values is changed due to additional metadata about each reply, see Document Context. But if we are using filters such as reply_str_value, it gets the reply object with value correctly. Moreover, for working with integration reply, the type values are renamed IntegrationValue -> IntegrationType and PlainValue -> PlainType for consistency.

2.10.X to 2.11.X¶

If we are using the questionnaire-report template, it is recommended to upgrade it to version 1.2.0 (from Registry or GitHub Release) so it displays also new Multi-Choice questions. Otherwise the choices won’t appear in the exported document if there are any.

2.9.X to 2.10.X¶

(nothing)

2.8.X to 2.9.X¶

(nothing)

2.7.X to 2.8.X¶

(nothing)

2.6.X to 2.7.X¶

(nothing)

2.5.X to 2.6.X¶

The document templates including the default questionnaire-report must be updated from https://registry.ds-wizard.org/templates.
Upgraded template metamodel version 2 requires manual migration of custom templates:
- questionnaireRepliesMap (map path:Reply) is no longer present in the context
- questionnaireReplies is now map with path:ReplyValue, provided filters (such as reply_str_value) are adjusted but wherever we used reply.value.value it should be reply.value with this change.
- Reply for item question is no longer an integer (number of answers) but a list of UUIDs representing the answers instead of integers. We added reply_items to extract the list from a ReplyValue.
Since 2.6.0, we are using WebSockets (for live collaboration). If we are using a proxy, we need to configure it accordingly. For example, in case of Nginx:

server {
   # ...

   location / {
      # ...

      # required for websockets
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_read_timeout 86400;
      proxy_send_timeout 86400;
   }
}

2.4.X to 2.5.X¶

Document templates have been moved from FS to database. To simplify the transition for custom templates, we added to the Docker image a script that loads templates from FS to the database via DSW API. But there are several new information that we need to provide in template.json file: id (instead of uuid), templateId, organizationId, version (semver), license, readme (Markdown). The id should be in format organizationId:templateId:version. Please note that this applies only for custom templates, default template can be removed from FS as it is added to the database automatically. The script must be enabled by setting envvar ENABLE_TEMPLATE_LOAD `` to ``1 and SERVICE_TOKEN according to the configuration.
Cron is no longer needed for the feedback synchronization (environment variables in docker-compose.yml) as DSW schedules synchronization internally.

2.3.X to 2.4.X¶

To unify configuration, document-worker now supports and prefers YAML configuration files.
Local/custom template.json files must be updated (renamed allowedKMs to allowedPackages, and several new attributes: description for template and shortName + color for each format).

2.2.X to 2.3.X¶

(nothing)

2.1.X to 2.2.X¶

Configuration of client and several features is now moved from application.yml file to in-app settings. therefore, it must be reconfigured during upgrade process. Additional secret must be configured in application.yml for encryption and JWT tokens (JWT.secret section has been removed), see Server Configuration configuration. It is recommended to first add general.secret (32 chars secret), start DSW, migrate options from application.yml in-app settings and then optionally clean up application.yml file.
User fields name and surname has been renamed to firstName and lastName - it needs be updated if used in custom mail or document templates.
Recommended version of MongoDB is updated to 4.2.3.

2.0.X to 2.1.X¶

There is a significant change related to new Document Worker that handles generation of documents from templates and filled questionnaires. We need to run RabbitMQ and document-worker with correct configuration according to server, see Deployment with Docker and Configuration Files for details.

1.10.X to 2.0.X¶

Changing the major version actually does not mean any problem in migration, it has been made due to significant internal changes (restructuring, new repositories, etc.)
If we are using Docker for running DSW, we need to change it according to new documentation of Deployment with Docker and Configuration Files.
Crontab image is no longer needed.
A DMP template configuration file must contain list of allowedKMs (see the default root template).

1.9.X to 1.10.X¶

Custom DMP templates needs to be upgraded to a new structure (see the default root template).