DSW User Guide#
The Data Stewardship Wizard (DSW) is a tool that helps researchers and data stewards create data management plans (DMPs) easily, efficiently, and in a FAIR manner.
Data stewards can easily capture the knowledge, including required project data and decisions in knowledge models that are then turned into per-project questionnaires to be filled by researchers. The questionnaires guide researchers through the process using recommendations, FAIR metrics indications, and only showing relevant questions based on previous answers.
Once the questionnaire is completed, a DMP can be easily generated using a selected template and output format. The document is then stored in DSW for easy access and future reference. This is especially helpful because many funding agencies now require a DMP for their application process.
But the benefits of using DSW go beyond just creating a DMP. Researchers also learn how to handle data correctly, make it FAIR, maintain it throughout the project, and curate it long-term. This intelligent, guided, and efficient approach to composing DMPs is useful for ELIXIR nodes, research institutions, and individual researchers alike.
Structure of the Guide#
The guide sections are organized into three categories:
About contains an introduction to the Data Stewardship Wizard and its content to gain quick insight into how it works at a high level.
Application is structured the same way as DSW’s main menu to quickly find the relevant sections about how to use a specific part of the application.
More contains all additional information related to DSW, such as development roadmap, how to develop own content, or how to run own instance.
Here are some recommended sections where to start based on the role:
Researcher |
Data Steward |
Admin |
|---|---|---|
Introduction |
Introduction |
Introduction |
Application |
Application |
Application |
More |
More |
|
Full Table of Contents
Introduction#
This section will cover the essentials of the Data Stewardship Wizard, including its key components and how they operate together to facilitate effective data management for researchers and data stewards.
Table of Contents
Overview#
Different components in the Data Stewardship Wizard are connected to create a data management plan and help with data management in general. Different components are typically made and used by different user roles. Data stewards work on preparing content (such as knowledge models or document templates) for researchers that they can use to work on their data management plans while filling in the questionnaires and exporting documents.
Different components of the Data Stewardship Wizard and roles interacting with them.#
Knowledge Model#
The knowledge model refers to something like a template for the questionnaire. However, it is not linear but has a tree-like structure with different branches based on the previous answer. Therefore, even if it can be complex overall, only the specific questions are used in the questionnaire.
Document Template#
While the knowledge model defines the structure of the questionnaire, it does not specify how the resulting document (such as the DMP) will look. We use document templates for that. They transform the answers into documents such as PDF, MS Word, or machine-actionable RDF. This way, we can only answer once and produce different documents.
Questionnaire#
A questionnaire is part of a project where researchers fill in their answers regarding their particular research. It uses a specific knowledge model that defines its structure.
Document#
Documents are produced from the questionnaire answers and a document template. The document template understands the knowledge model structure and knows how to transform the questionnaire answers into a specific document in the selected format. The documents are saved within a project where they were created from the questionnaire.
Knowledge Model#
The knowledge model is a tree-like structure of chapters, questions, answers, and other entities that servers as a template for the questionnaire. All the different questions, their possible answers and follow-up questions, metrics and more is defined there.
While all the possibilities are defined in the knowledge model, when researchers use it to create their project, they don’t see everything, but only the top-level questions and more detailed questions are only asked if relevant to their use case.
Knowledge models are created by data stewards in the knowledge model editor.
Knowledge Model Structure#
Knowledge model consists of several entities connected together. You can see how they are connected in the following diagram and read more details about them below.
Knowledge model schema#
Knowlege Model#
At the top level, the knowledge model contains chapters, and entities refered to elsewhere from the knowledge model: metrics, phases, question tags, and integrations.
Chapter#
The knowledge model consists of chapters at the top level. Each chapter has a name, a description and a list of questions. Usually, chapters are used to group the questions on the same topic together.
Question#
Questions are used to collect the answers from users. Each question has a title (the actual question), a description, a phase when it becomes desirable, a list of references and experts, and a selection of question tags.
Then there are some additional settings based on the question type.
Options Question#
The options question contains a closed list of answers where users can pick one. Answers can have some follow-up questions that are only presented to the user when they pick the answer. So the options question can be used for questionnaire branching.
List Question#
The list question is used when there are multiple answers and we want to ask more details about those. For example, we can ask about different datasets that will be produced – users will have multiple datasets but we want to ask the same questions for each of those. For that, we configure the item template, which defines the questions for each item.
Value Question#
The value question asks for a single value that users type in. There are many different types of the value question that can be used:
String
Number
Date
Date Time
Time
Text
Email
URL
Color
The input field differs based on the value type (simple input for string, date picker for date, etc.). Some of these have a check whether the entered value is valid (such as valid email or URL) and displays a warning if not.
Integration Question#
The integration question is connected to an external resource where the users can pick the answer from. We need to select an integration that the question uses and sometimes additional properties, based on the integration configuration.
Users can then search the external resource through the questionnaire and choose the answer. The advantage is that the answer is not only the text but also a link or PID of the selected item making it more FAIR.
If the desired answer is not present in the external resource, users can still fill in a text answer themselves.
Multi-Choice Question#
The mutli-choice question has a list of choices. Users can then pick as many of those choices as they wish. There are, however, no follow-up questions available for this question type.
Answer#
An answer is used with options questions. It contains a label which is the answer itself. Then an advice which is visible only if the answer is selected. We can use this when users pick answer that is not great to provide them further guidance on how to improve.
Answers can have follow-up questions that are only visible if the answer is selected. We can use this to ask only relevant questions based on the previous answers.
If there are some metrics created in the knowledge model, we can configure how each answer affects them. The result for each metric is eventually calculated as a weighted average of all answers affecting that metric. Therefore, we need to configure:
weight [0..1] - how important the answer is (0 = not important at all, 1 = very important)
measure [0..1] - how it affects the metric (0 = bad, 1 = good)
Choice#
A choice is used with mutli-choice questions. It only contains a label which is presented to the user.
Reference#
We can provide some additional references for questions to help users better understand it or learn more details. There are more types of references.
URL Reference#
A URL reference is a simple link to any website. It has URL which is the actual link and a label that describes what the reference is about.
Book Reference#
Warning
Book references are deprecated.
Resource Page Reference#
Warning
Resource page references are not yet implemented.
Expert#
We can provide a contact information to an expert for some questions. An expert has a name and an email. We can use this, for example, if there is an expert for a specific topics in our institution and we want to make it easy to find out in our customized knowledge model.
Metric#
We can define metrics for each knowledge model based on our needs. Each metric has a title, an abbreviation, and a description. Once the metric is defined, we can configure which answers affect it and how.
This can be use, for example, to define the FAIR metrics:
F - Findability
A - Accessibility
I - Interoperability
R - Reusability
And then define which answers affect which FAIR metrics to provide more feedback to the researchers.
Phase#
We can create phases to reflect the workflow. Such as: Before submitting the proposal, Before submitting the DMP, etc. Each phase has a title and a description.
Once we have phases defined, we can assign them to questions to indicate where each question become desirable. The phases implicitly follow the order in which they are in the knowledge model and the question is considered desirable from the defined phase and on. So for example, if a question is desirable in Phase 2, it is implicitly desirable in Phase 3, Phase 4, etc.
Question Tag#
We can define question tags on the knowledge model and then assign them to different questions. This can be used to group together questions on the same topic or for the same purpose.
When researchers create a new project from the knowledge model, they can only choose the question groups they are interested in for their research. So we can use this to create a very rich knowledge model but researchers will be able to use only the parts relevant to them.
Integration#
Integrations define a connection to an external service or resource where we can get the answers from. They are used with integration questions. For each integration we configure some basic information, such as ID, Name, or Logo URL. Other configuration varies based on the integration type. More information about how to configure integration is available under the integration questions documentation.
API Integration#
API integration connects to an external service API to search for the answers. We need to provide some request and response configuration, so DSW can use the API.
Widget Integration#
Widget integration doesn’t use an API but a widget implemented using the DSW Integration Widget SDK. Then we need to configure the widget URL where the widget is deployed.
Annotations#
Annotations are arbitrary key value pairs that can be assigned to any entity in the knowledge model. These can provide some additional information for the document templates.
Knowledge Model Customizations#
A knowledge model doesn’t have to be created from scratch. Instead, it can be created as a customization of an existing knowledge model.
We can choose any existing knowledge model and customize it to our needs. We can add, modify, or remove any entities. If there are newer changes in the parent knowledge model, it is possible to get them into our child knowledge model using the knowledge model migration.
Document Template#
Document templates transform the answers from a questionnaire to a document of a specific format. The document template usually follow some standard template, such as Horizon Europe DMP, and can support different formats, such as PDF or MS Word. The formats can be basically any text-based format, so it can also be for example a JSON or XML. It follows that we can easily use them to crate a machine-actionable output, too.
Document templates can transform the answers from questionnaires to different formats, such as human-readable PDF or machine-actionable JSON.#
It is important to note that the document template always define what knowledge models it is compatibly with because in order to transform the answers to the document it needs to understand what the questions are and how the answers should be composed into a document.
We can get existing document templates in the DSW Registry, or create our own using document template editor or DSW Template Development Kit (TDK).
Project#
A project is a central part of where we work on our data management plans. It is based on a knowledge model and uses one or more document template for generating documents.
Projects are fully collaborative, so we can share them with other researchers and work together.
Questionnaire#
The questionnaire is part of the project where we fill in our answers. It is generated based on the selected knowledge model. It shows only the questions that are relevant based on our previous answers. There are many more features to provide guidance and embrace collaboration, such as FAIR metrics, comments, TODOs, or version history.
Documents#
The outcome of our endeavors in DSW should be the data management plan, the document. Once we have enough answers to the questionnaire, we can generate a document using a document template. These documents are saved within the project. We can create as many as we wish with different document templates and formats. We can also set up a default document template for the project and quickly see a preview of the document.
Knowledge Models#
This section covers how to manage existing knowledge models and how to create new ones and make them available for researchers.
Table of Contents
Knowledge Model List#
As data stewards and admins, we can manage the knowledge models that are in our DSW instance. Then, researchers are using and browsing the knowledge models. We can access the list of knowledge model from the main menu via Knowledge Model. The list can be filtered and sorted by name or creation date.
For each knowledge model (KM), we can see the latest version in the list. If we want to read more about a specific KM or see the older versions, we need to access the Knowledge Model Detail by clicking the name of KM or clicking Open from the right item menu (three dots). There are also other options for each item:
Preview to see how Projects generated using this KM would look like.
Export for exporting the latest version of the KM as a file.
Create KM editor is a shortcut for Create Knowledge Model Editor for creating a new version.
Fork KM is again a shortcut for Create Knowledge Model Editor for to create a fork (some more specific KM based on this one).
Create project is a shortcut to Create Project with this KM.
Set deprecated or Restore for setting a KM deprecated when we no longer want the researchers to use it.
Delete for all versions of the KM (possible only if is not used in any projects or linked in other KMs and editors).
Note
The options in the item menu are based on the role of a current user, e.g. a researcher cannot create KM editor.
For data stewards and admins, update available may appear if there is a newer version of the knowledge model in the DSW Registry (and if configured).
Finally, there is an option to Knowledge Model Import by click the Import button in the top right part of the screen.
List of all knowledge models with actions.#
Table of Contents
Knowledge Model Import#
We can import an existing knowledge model by navigating to Knowledge Model List () in the main menu and then clicking on Import button.
From DSW Registry#
If the DSW instance is connected to the DSW Registry, it is possible to import knowledge models from it by entering the knowledge model ID of desired template (e.g. dsw:lifesciences:2.4.0) and pressing the Import button.
Note
In case of knowledge model present in the DSW Registry, we will be notified about the available upgrades.
Input for importing a knowledge model from DSW Registry.#
From file#
We can import a knowledge model as a KM file. Such a file can be created as an export from DSW (from Knowledge Model List or Knowledge Model Detail).
Input for importing a knowledge model using a KM package.#
Knowledge Model Detail#
We can visit a knowledge model detail by clicking on a desired KM in the Knowledge Model List (or selecting View detail from the right item menu). The detail shows basic information about the knowledge model such as its name, ID, version, license, metamodel version, or (if applicable) what is the parent knowledge model).
The main part of the detail is the README of the KM that should contain basic information and changelog. In the right panel under the basic information, we can navigate to other versions of the KM or navigate to the DSW Registry (if the KM is present there).
In the top bar, we can Export the knowledge model as a KM file or Delete this version of the knowledge model (only if it is not already used for some projects or other KMs and editors).
In the top pane, we can see the options based on our role:
Preview can be used to check the content of the KM via the Knowledge Model Preview feature.
Export for exporting the latest version of the KM as a file.
Create KM editor is a shortcut for Create Knowledge Model Editor for creating a new version.
Fork KM is again a shortcut for Create Knowledge Model Editor for to create a fork (some more specific KM based on this one).
Create project is a shortcut to Create Project with this KM.
Set deprecated or Restore for setting a KM deprecated when we no longer want the researchers to use it.
Delete the specific version of the KM (possible only if is not used in any projects or linked in other KMs and editors).
If we are not seeing the latest version of the KM, a warning message is shown in the top. Similarly, we will see a notification that update is available if there is a newer version in the DSW Registry (if configured).
Detail of a knowledge model.#
Knowledge Model Preview#
The preview feature for knowledge models allows us to check the content, i.e. how the questionnaire looks like. It can be used to navigate just through the top-level questions. However, with a know UUID of the question, we can create a direct link to a certain question:
https://researchers.ds-wizard.org/knowledge-models/dsw:root:2.4.4/preview?questionUuid=0b12fb8c-ee0f-40c0-9c53-b6826b786a0c
For a nested question, all needed question above in the tree will be automatically prefilled. The KM preview may be also available to non-logged-in users, if configured by administrators in Knowledge Models Settings.
A user can directly create a new project from the KM preview. Again, this can be available also for non-logged-in users if anonymous projects are enabled by administrations in Projects Settings.
Knowledge Model Editors#
Here, we can see a list of all knoweldge model editors. Everyone with the data steward role assigned can see all the knowledge model editors.
List of knowledge model editors.#
We can use the search field to search for a specific KM editor. The editors are sorted by when they were last updated but we can change that.
We can create a new knowledge model editor by clicking the Create button.
By clicking the triple dots on each of the item in the list we can access some actions:
Open Editor - simply open the editor detail
Upgrade - if there is a newer version of parent knowledge model, we can use upgrade action to start a knowledge model migration, otherwise the action is not visible
Publish - to publish a new version of the knowledge model
Delete - to delete the knowledge model editor (cannot be undone)
If there is an ongoing knowledge model migration, there are different actions:
Continue migration
Cancel migration
Table of Contents
Create Knowledge Model Editor#
We can create a new knowledge model editor by navigating to in the main menu and then clicking the Create button.
Form for creating a new knowledge model.#
Every knowledge model needs to have a name, a knowledge model ID and version. The name should be something descriptive to help users understand what the knowledge model is about. The knowledge model ID is used for the identification together with the organization ID and knowledge model version after it is published. So the identifier of the knowledge model is:
<organizationId>:<knowledgeModelId>:<version>
We can create a new project either from scratch, i.e. the new knowledge model will be empty and we will build it all ourselves, or based on an existing knowledge models, which means that everything from the chosen knowledge model will be copied to ours. We can start from there and add, delete, or modifiy the existing entities in there. We just need to choose the original knowledge model in the based on field. Alternatively, we can open the knowledge model detail and click on Fork KM there.
We can only have one knowledge model editor with the same knowledge model ID. If we deleted the editor but want to continue working on that knowledge model, we can create a new editor with the same knowledge model ID. Or we can open the knowledge model detail and click on Create KM editor there to have the editor create form prefilled.
Knowledge Model Editor#
Knowledge model editor is where we build knowledge models. In this section, we will see what entities we can add there, how they are connected, how to work with the editor and how to publish the knowledge model.
Knowledge model editor.#
Table of Contents
Knowledge Model#
The Knowledge Model tab is where we work on the entities (such as questions or answers) that appear in the knowledge model. We define the structure here.
Navigation#
In the top row, we can see the breadcrumbs that show us where we are within the knowledge model. This allows us to quickly jump back up in the hierarchy.
On the lefthand side of the editor, there is a navigation tree reflecting the structure of the knowledge model. We can click on the arrows to expand or collapse individual entities, or click on Expand all or Collapse all. By clicking on the entity name, we open its editor.
Knowledge model editor navigation.#
Tip
There is no search, however, if we want to quickly find an entity by name we can click on Expand all in the navigation tree and use search functionality of our web browser.
Editors#
The main area of the Knowledge Model tab is the actual editor. The form fields change based on the entity we edit, but there are some shared actions:
Editor action buttons.#
Copy UUID - every entity has a generated UUID, we can use this button to copy it. We usually need it for document template development.
Move - we can move entities around the knowledge model. However, not every entity can be put under everything. We can open the move modal window and see where the current entity could be move to.
Delete - delete is simply used for deleting the entities. This action cannot be undone, so we need to be careful what we delete.
There are different entities we can edit in the knowledge model, the editor shows different fields based on what we edit:
Example of question editor form.#
Besides their own fields, each entity has so called Annotations. They are arbitrary key value pairs that can be assigned to the entity and used later, when developing a document template.
Warnings#
The editor checks for some possible problems, such as empty title for a chapter or no answers for an options question. If there are any, the Warnings tab appear and we can quickly navigate to those problems and fix them.
Warnings in the knowledge model editor.#
Phases#
If there are any project phases set up in the knowledge model, we can use the Phases tab to see an overview of which questions were assigned to be completed in each phase and to change the phase assignments.
We can simply click to the checkbox in the corresponding phase - question table cell to assign/unassign the phase to that particular question.
Phases editor where we can assign questions to phases.#
Question Tags#
If there are any question tags set up in the knowledge model, we can use the Question Tags tab to see the overview of what questions were these tags assigned to, and to assign the question tags to the questions.
We can simply click to the checkbox in the corresponding question tag - question table cell to assign/unassign the question tag to that particular question.
Question tag editor where we can assign question tags to questions.#
Preview#
We can use Preview tab to quickly see how the resulting questionnaire would look like. If there are any question tags set up, we can also try different combinations of selected tags and observe the changes in the questionnaire.
Preview of the resulting questionnaire in the knowledge model editor.#
Settings#
The Settings tab, allows us to adjust atributes of the the knowledge model:
Name - should be short name of the Knowledge Model.
Description - this should be really short and descriptive.
Knowledge Model ID - is an unique Knowledge Model identifier.
Version - this is a number indicating which is the latest version of the Knowledge Model, because it can change over time.
License - this is used when we want to share the knowledge model with other people so they know how they can do that. We recommend using a license identifier from SPDX Licens List.
Readme - this is where we can describe everything we need about the knowledge model. We can, for example, include a changelog of what changed in which version, etc. We can use Markdown in this field to provide some nice formatting.
Note
Name, Description and Version are all visible to the researcher, when they select a Knowledge Model for their project. So the Name and Description should provide them with enough information to select a correct one.
Knowledge Model ID together with the organization ID and knowledge model version after it is published are used for the identification. So the identifier of the knowledge model is:
<organizationId>:<knowledgeModelId>:<version>
For the version number we recommended using similar approach as in semantic versioning. So when we have a version <major>.<minor>.<patch>, change in the major version number indicates some breaking changes (deleting questions, significant changes in the questionnaire structure, etc.), change in minor version number indicates some new changes that are backwards compatible (i.e., adding a new question), and change in the patch version number indicate some fixes (such as fixing some typos).
If the knowledge model was based on another knowledge model, we can also see the Parent Knowledge Model in the settings.
In the Danger Zone we can delete the knowledge model. Once it is deleted it can no longer be recovered.
Knowledge Model settings.#
Publish#
Before the knowledge model can be used by researchers in their projects, we need to publish it. We can do that by clicking the Publish button in the top right corner of the Knowledge Model Editor.
If we click the button, we are prompted with the metadata details to check them before publishing. We cannot change anything here, so if we want to change it, we have to press Cancel and edit the details on the Settings tab of the Knowledge Model Editor.
Publish dialog where we can confirm or cancel publishing of the Knowledge Model.#
If we confirm the publishing of the Knowledge Model by clicking Publish in the modal window, the Knowledge Model becomes available to all users and is accessible in Knowledge Model List.
The Knowledge Model Editor will remain in the Knowledge Model Editors list and will be available for any future changes.
Knowledge Model Migration#
Knowledge model can be created either from scratch or based on existing one and further modified. If that happened, we can call the original Parent KM and the modification Child KM. We can create the Child KM any published version of the Parent KM, make some modification and publish a version of the Child KM. However, the Parent KM can evolve, and at some point we might want to have those changes in our Child KM, too.
That is what the knowledge model migration is for. Once a new version of the Parent KM is published, we can start the KM migration where we go through these changes. We can choose whether we want to apply or reject these modification to our Child KM during migration. At the end, we publish a new version of the Child KM with all the selected changes.
Schematic representation of KM migration.#
Creating a Knowledge Model Migration#
We can start the knowledge model migration from the list of knowledge model editors. If there is a newer version of the Parent KM available for particular knowledge model editor, we can see the update available badge next to the name of the editor.
Badge indicating that there is a newer version of the Parent KM.#
We can either click on the badge directly, or choose Upgrade option from the dropdown menu. It will open a modal window where we can choose a new version of the Parent KM that we want to migrate our Child KM to. Usually, we want to pick the latest.
Modal window to create a new knowledge model migration.#
Knowledge Model Migration#
During the migration, we can see all the changes one by one and can choose whether we want to Apply or Reject the change. We can also choose to Apply all if we simply want everything.
During the migration we can apply or reject the changes form the Parent KM.#
Cancelling a Knowledge Model Migration#
We can cancell the knowledge model migration at any point before we publish the new version of the Child KM. We need to navigate to the list of knowledge model editors and choose Cancel migration from the dropdown menu for the desired KM editor.
Finishing a Knowledge Model Migration#
After we resolve all the changes, we are ready to publish the new version of the Child KM. To do that, we need to click on the Publish → button. This will open the Publish new version screen where we need to provide additional information for the new version of the Knowledge Model.
The publish screen shows us some information about the knowledge model, such as it’s Knowledge Model Name and Knowledge Model ID. We need to choose the new version number.
Note
We recommended using similar approach as in semantic versioning. So when we have a version <major>.<minor>.<patch>, change in the major version number indicates some breaking changes (deleting questions, significant changes in the questionnaire structure, etc.), change in minor version number indicates some new changes that are backwards compatible (i.e., adding a new question), and change in the patch version number indicate some fixes (such as fixing some typos).
Then we need to add some additional metadata (these fields are pre-filled if there was a previous version published):
License - this is used when we want to share the knowledge model with other people so they know how they can do that. We recommend using a license identifier from SPDX Licens List.
Description - this should be really short and descriptive. It is used, for example, in select boxes when creating a new project to help researchers choose the best knowledge model for their use case.
Readme - this is where we can describe everything we need about the knowledge model. We can, for example, include a changelog of what changed in what version, etc. We can use Markdown in this field to provide some nice formatting.
Document Templates#
This section is about managing the document templates and document template editors. Similarly to Knowledge Models, we can manage Document Template List or work on new or customise existing templates using Document Template Editors.
Table of Contents
Document Template List#
As data stewards and admins, we can check and manage all document templates from the list accessible from the main menu via Document Templates. The list can be filtered and sorted by name.
For each document template, we can see the latest version present; however, we can see all the versions by accessing the Document Template Detail by clicking the name of the template or via View detail from the right dropdown item menu. The dropdown menu also offers Export of the latest document template or Delete of entire template (all its versions).
Note
A document template can be deleted only if it is not used already for documents, projects, or settings.
Each item may be marked with unsupported metamodel if the document template is not compatible with the version of DSW. Also, update available may appear if there is a newer version of the template in the DSW Registry (and if configured).
Finally, we can use Import new document templates by clicking the top right button (see Document Template Import for details).
List of all document templates with actions.#
Table of Contents
Document Template Import#
We can import an existing document template by navigating to Document Template List () in the main menu and then clicking on Import button on the list of document templates.
From DSW Registry#
If the DSW instance is connected to the DSW Registry, it is possible to import document templates from it by entering the document template ID of desired template (e.g. dsw:questionnaire-report:2.7.1) and pressing the Import button.
Note
In case of document templates present in the DSW Registry, we will be notified about the available upgrades.
Input for importing a document template from DSW Registry.#
From file#
We can import a document template as a ZIP package. Such a package can be created as an export from DSW or using the Template Development Kit (see Document Template Development).
Input for importing a document template using a ZIP package.#
Document Template Detail#
We can check a document template detail by clicking on a desired template in the Document Template List (or selecting View detail from the right dropdown). The detail shows basic information about the template such as its name, ID, version, license, metamodel version, or supported document formats.
The main part of the detail is the README of the template that should contain basic information and changelog for the template. In the right panel under the basic information, we can navigate to other versions of the document template, navigate to the DSW Registry (if the template is present there), or check compatible knowledge model with the template.
In the top bar, we can Export the template as a ZIP package or Delete this version of the template (only if it is not already used for some documents). We can also quickly navigate to Create Document Template Editor by clicking Create editor; it will prepare editor creation for a new version of this document template. Finally, there is the possibility Set deprecated which will change the state of the document template so it is no longer usable by researchers in their projects (it becomes unavailable).
If we are not seeing the latest version of the template, a warning message is shown in the top. Similarly, we will see a notification that update is available if there is a newer version in the DSW Registry (if configured).
Detail of a document template.#
Document Template Editors#
On this page, we can see a list of all document template editors. Everyone with the data steward role assigned can see all the document template editors and use them.
We can use the search field to search for a specific document template editor. The editors are sorted by when they were last updated but we can change that.
We can Create Document Template Editor by clicking the Create button.
By clicking the triple dots on each of the item in the list we can access some actions:
Open Editor - simply open the Document Template Editor (we can also click the name of the editor)
Delete - to delete the document template editor (cannot be undone)
List of document template editors with actions.#
Table of Contents
Create Document Template Editor#
We can create a new document template editor by navigating to in the main menu and then clicking the Create button.
Form for creating a new document template editor.#
Every document template needs to have a Name, a Document Template ID and version. The name should be something descriptive to help users understand what the document template is about. The Document Template ID is used for the identification together with the organization ID and document template version that we have to fill as a New Version. So the identifier of the document template is:
<organizationId>:<documentTemplateId>:<version>
We can create a new editor either from scratch, i.e. the new document template will be empty and we will build it all ourselves, or based on an existing document template, which means that everything from the chosen document template (files as well as configuration) will be copied to ours. We just need to choose the original document template in the Based on field. Alternatively, we can click on Create editor from Document Template Detail.
We can only have one document template editor with the same document template ID. If we deleted the editor but want to continue working on that document template, we can create a new editor with the same document template ID. Alternatively, we would have to use a different document template ID.
Document Template Editor#
A document template editor allows us to edit both configuration and all files of a document template. We can manage the configuration (such as template metadata, formats, and steps) on the Settings tab. The files including directories can be managed (created, edited, deleted, or uploaded) on the Files tab. Finally, the Preview tab allows us to quickly check how the document template works for a certain project and format and how the resulting document looks like.
There is also option to Publish the document template via Publish buttom in the top right corner.
Document template editor.#
Table of Contents
Files#
The Files tab in Document Template Editor allows us to manage the files and directories (or folders) of the document template. There is a file tree on the left side whereas the main part is containing the built-in text editor. We can create a new folder or text file by clicking Add on the of top the file tree and then File or Folder. Once we enter a desired name, the folder or file is created in the active directory. Alternatively, we can upload a file by clicking Add and then Upload.
If we select a text file in the file tree, it is opened in the built-in text editor. We can also open multiple files (the editor supports tabs). Moreover, if we have more files opened, we can see a split view icon on the top of the file tree. When we click it, it will split the view and move the opened file to the other group. If there are already two groups, it switches the file between the two groups. We can close the file by clicking cross icon in its tab.
We can easily delete a file or a folder; when it is selected in the file tree, we can simply click the red trash icon above the file tree. The deletion must be confirmed in the prompt so we will not delete something by accident as it is not reversible operation.
If we make some changes in a file, the asterisk (or star) symbol will appear by the name in the tree view as well as in the tab (if opened). The changes must be then saved using Save button or discarded using Discard button in the top right corner. Those buttons will appear instead of Publish as it is not possible to publish a document template with unsaved changes. After saving the changes and switching to Preview (or refreshing it), the document will be re-generated using the newly changed document template.
Files editor with tabs and split view.#
Preview#
The Preview tab of Document Template Editor allows us to quickly check how our document template works and how the resulting document looks like for a selected project and document format. We can select any Project that we are allowed to view. Then, we select the desired Format (one of those specified on the Settings tab of the editor). Any change in the template configuration and its files will trigger re-generation of the preview (we can simply click the Preview tab again).
Preview in a document template editor (without selected Project and Format).#
Settings#
The Settings tab of Document Template Editor allows us to adjust the configuration and metadata of the document template. It is split to the following three parts:
General#
This part allows us to change the metadata about the document template:
Name should be short name of the template.
Description should be short and descriptive (users will see it while selecting a document template).
Template ID is the document template ID (as explained for Create Document Template Editor).
Version of the document template.
License should contain a name of used license (e.g. Apache-2.0 or unlicensed).
Readme can contain a longer description, acknowledgements, notes how to use the template, links to resources, and a changelog.
Knowledge Models#
We can specify here what knowledge models are compatible with our document template. This is useful to capture if our template is usable only with a certain knowledge model(s) which will guarantee us some content (specific chapters, questions, answers, etc.). For each entry, we are prompted to specify the Organization ID, Knowledge Model ID, Min Version, and Max Version. Any of these value can be left empty which means any value.
For example, if we set Organization ID to dsw, Knowledge Model ID to root, Min Version to 2.4.0 and leave Max Version empty, it will mean knowledge model dsw:root:2.4.0 and any higher version. So that example would work for dsw:root:2.4.4 or dsw:root:2.5.0 but not for dsw:root:2.3.0 nor acme:base-km:2.6.0
Allowed knowledge models specification for a document template.#
Formats#
Each document template can support multiple file formats and users will be able to select which one they want to use to generate (or preview a document). We can add a new format by clicking Add button. Then, each format must have a Name and Icon (by using Font Awesome).
Each format has some steps which capture how a file for that format is produced. There are different steps defined such as json, jinja2, pandoc, or wkhtmltopdf which is used as its Name. Then, the step may have certain configuration Options. For example, jinja2 must have content-type, extension, and template specified. All the possible steps and their options are further described in the Document Template Development.
Markdown format specification with jinja step.#
Publish#
Once we are ready with our document template in the editor, we can publish a new version by using the Publish button located in the top right corner of Document Template Editor. If we click the button, we can check the metadata details and confirm the publishing. We cannot change anything here, so if we want to make some changes, we have to press Cancel and edit the details on the Settings tab of the Document Template Editor.
If we confirm the publishing of the document template by clicking Publish in the modal window, the document template becomes available to all users and is accessible in Document Template List. Moreover, the document template editor disappears (as the state of the document template changed). If we want to directly continue in developing a new version, we have to Create Document Template Editor.
Confirmation of document template publishing.#
Projects#
This section is about projects – how to create them, how to work and collaborate on them, how to generate documents, how to upgrade projects, what are the project templates and more.
Projects are mainly used by the researchers. We have the introduction video about how to create project, fill the questionnaire and get the documents.
Table of Contents
Project List#
In the project list, we can see a list of all projects we have access to. Those are the projects where we were assigned to with any role or that are visible by all other logged-in users in the project sharing settings. The projects are filtered to those we are explicitely assigned to by default.
Project list with filters.#
We can search for specific projects using the search field or filter them using additonal filters:
Project Template - if we want to see only project templates or regular projects
Project Tags - show only the projects that has specific tags assigned
Knowledge Models - show only the projects created from a specific knowledge model
Users - filter only the projects with specifc users assigned to them
Note
Note that some of the filters can be disabled based on the DSW instance settings or user role.
We can create a new project by clicking the Create button.
By clicking the triple dots on each of the item in the list we can access some actions:
Open project - will simply open the project
Clone - will create the exact copy of the project
Create migration - will start a project migration
Delete - will delete the project (this action cannot be undone)
If there is an ongoing project migration, there are different actions:
Continue migration - to come back to the ongoing project migration
Cancel migration - to cancel the ongoing project migration
Table of Contents
Create Project#
We can create a new project by navigating to Projects in the main menu and then clicking on Create button on the project list.
Note
If we have Researcher role we also have a Create Project widget on our dashboard right after logging in. We can click on Create button there, too.
Based on our instance configuration, we can create the new project from a project template, custom, or both.
Different options how to create a project.#
From Project Template#
There are many options how to create and configure a project, such as what knowledge model or document template to use. Project templates are prepared projects by data stewards with possibly pre-selected knowledge models and document templates, they can have some pre-filled answers, comments and TODOs as well.
So if there are any project templates created in our instance, we can use them to have a smoother start of our project. We just need to give our project a name and choose the project template from offered options.
Custom#
If there are no project templates available or we don’t want to use them, we can choose to create a custom project. We need to give our project a name again, but this time we choose a knowledge model from offered options.
This will create an empty project with only the selected knowledge model and we need to configure everything (such as a document template) ourselves.
Project Detail#
Project detail is where we work on our data management plan. In this section, we explore different features from filling in the answers to collaborating with other people or generating documents.
Project detail opened on the questionnaire page.#
Table of Contents
Questionnaire#
Questionnaire is the first tab of the project detail. This is the most imporant part where we fill in all the details about our project.
Current Phase#
If the knowledge model we use for the project has phases defined, we can see a phase selection in the questionnaire detail. Different questions become desirable based on the selected phase. For example, some should be answered before submitting the proposal, while others can be filled later.
Phase selection widget.#
By selecting the phase we are currently in, we can see the desirability of questions. We can also see the number of questions that still need to be answered in this phase for each chapter in the chapter list.
There are three desirability states the question can be in:
red, with a pen icon - this question must be answered in the current phase
light grey, with an hourglass icon - this question will have to be answered in some later phase
green, with a checkmark icon - this question has already been answered
Note
If there is no phase defined on the knowledge model, the current phase selection is not visible in the questionnaire detail.
Chapters#
Below the current phase selection, we have a list of chapters. We can see the number of questions that are yet to be answered in current phase (or overall if there are no phases defined in the knowledge model). We can use this list to navigate freely between chapters.
For the opened chapter, we can see a navigation tree for the chapter structure, showing the questions, follow-up questions, items, etc. We can use this tree to quickly navigate to a specific question in the chapter.
Chapter list showing the also the questions for the opened chapter.#
Questionnaire Area#
The questionnaire area fills the most space in the questionnaire screen. It displays the questions and answers from the opened chapter.
Each question has an identifier which indicates the chapter it belongs to, as well as its order and nesting within the chapter. For example, I.1.a.5, where the Roman numeral represents the number of the chapter, and the remaining numbers indicate the order and nesting of the question. Then there is also the question name.
Some additional information can also be part of the question:
Question tags - can indicate some additional groupping of questions, for example what DMP templates is this question used for
Description - additional information explaining the question
Desirability - what phase this question become desirable in
List of references - links to additional external resources related to the question
List of experts - whom to contact when help is needed with answering the question
Based on our role in the project and specific instance settings there are some additional actions besides answering the question:
Provide feedback for the question
We can get more information on how various collaboration tools work and can be used in Sharing.
The most important part is, however, answering the question. The way of how to answer the question differs based on the question type.
The following video tutorial explains questions and different question types in more detail.
Options Question#
Options question has a list of pre-set answers and we can choose one from those. There can be some follow-up questions (indicated by the icon by the answer). These questions are displayed only if we select that answer.
If there are some merics set for the answers, we can see labels with the metrics by the answer as well. The color of the label indicates how good or bad the answer is (red means bad, green good, yellow something in between).
Options question with a closed set of answers.#
List Question#
List question doesn’t have a simple answer but a list of items. Each of the items has the same set of subquestions. For example, a list question asking about the project contributors where each item represent one contributor with questions about their name, role, etc.
We can simply click on Add button under the question to add a new item. Then, we can answer the questions for the item. If the item has a lot of questions, we can use the arrow icon in the item’s top left corner to fold/unfold the item.
There is a trash bin icon in the item’s top right corner that we can use to delete the item. If there are more than one item, there are also arrow icons that we can use to change the order of the items.
List question with a single item.#
Value Question#
Value question contains an input field for our answer. This can be a simple text field (such as asking for a project contributor’s name), or some additional widget, for example a date picker.
Some of the value types contains a validation (e.g., email or URL). We can still type in an invalid answer, but it will display a warning and also show it in the list of warnings.
Value question with a simple text input.#
Integration Question#
Integration question is connected to an external resource where it searches for the answers. The input field works as a search field, so when we start typing something, it will search the external resource and offers us a list of possible answers.
When we pick an answer from the list, we not only have the answer but also a link to the selected item in the external service. If the answer we searched for is not there, we can simply keep what we have written in the input field. We just won’t have the link with this answer.
Integration question with a response from FAIRsharing containing also a link.#
Multi-Choice Question#
Multi-choice question is similar to the options question, however we can choose more there one answer and there are no follow-up questions.
Multi-choice question with many choices.#
View settings#
In the questionnaire tab, there is a menu bar with various options. The first one is View, where we can show or hide some question details:
Answered by - show/hide who and when answered questions
Phases - show/hide what phase the questions are desirable in
Question tags - show/hide the question tags
Non-desirable questions - show/hide questions that are not desired to be filled in current phase
Metric values - show/hide value of metrics for accessibility purpose
Different view options accessible form the questionnaire toolbar.#
Import answers#
Questionnaire answers can be imported for various sources using project importers.
If there are some project importers available for the project, there is the Import answers button in the questionnaire menu bar. We can choose one of the available importers there and then follow the instructions in the importer window.
Warnings#
Some value questions (such as email or URL) validates the answer written there. If it is an invalid value, we will see Warnings tab in the questionnaire menu bar with a badge showing the number of warnings. If we click on it, we can see a list of all questions that has a warning and we can click on it to navigate quickly to that question.
Warnings refering to questions with invalid values.#
TODOs#
When we are filling in the questionnaire, we can stumble upon a question that we don’t know how to answer yet, but we don’t want to forget to come back to that question. We can click on + Add TODO to add a TODO to the question.
We can then open the TODOs tab from the questionnaire menu to see the list of all questions with assigned TODO in the questionnaire. By clicking on a question there, we can quickly jump back to that question and fill it.
List of TODOs.#
Version History#
When we open the Version history tab from the questionnaire menu bar we can see the list of all the changes that happened in that questionnaire. We can see who and when made what changes groupped by months and days.
Version history shows all events changing the project.#
Name a Version#
At any point (also retroactively) we can name a version. Click on the triple dots on any event we want to name and choose Name this version. Then we just fill in name and description of that version. If the event already has a named version, we can choose Rename this version instead.
When we have some named versions, we can choose Named versions only. Then, we don’t see every single change but only the important versions we gave a name to.
View Questionnaire in a Version#
Thanks to the version history, we can see how the questionnaire was filled at any point in the past. We simply find the event in the version history and choose View questionnaire from the event menu.
Create Document from an Older Version#
Sometimes, we might want to create a document from an older version. For example, we created only a PDF document, but later we find out that we also needed a Word document. To do that, we simply find that version in the version history and select Create document. Then, we just fill in the details in the form and create the document.
Revert to an Older Version#
We can also revert a questionnaire to an older version. We can simply find the desired version in the version history and choose Revert to this version from the event menu.
Warning
Reverting to an older version cannot be undone. It is therefore recommended to create a copy of the project before reverting.
Metrics#
In the Metrics tab in the project detail we can see a Summary Report for the whole questionnaire and then the same details for each individual chapter.
Summary report of project metrics.#
The report shows how many answered questions out of how many are there for the current phase (if there are phases in the knowledge model) and overal.
If there are any metrics in the knowledge model, the report also shows the score for each metric. The score is calculated as a weighted average of all the answers affecting that metric and is always between 0 and 1. If there are at least 3 metrics present, a spider chart is also displayed.
There is also a metrics description at the bottom of the page to better understand what exactly each metric means.
Preview#
We can quickly see how the resulting document would look like in the Preview tab.
Document preview in the project detail.#
Default Document Template Not Set#
The preview uses the default document template. If there is no default document template set in the project settings, we need to set it first.
Preview only works when the default document template is set.#
Download Preview#
Not all formats can be displayed in the web browser, for example a MS Word document. In that case, we’ll be able to download the document instead of previewing it.
Some formats cannot be displayed in the web browser.#
Documents#
In the Documents tab in the project detail we can manage the documents related to the project. We can see a list of all the documents with their format, file size, and document template used to create that document.
While we can quickly see how the current state of the questionnaire looks like in the document in the preview tab, the documents created here are persistent. That means that the once the document is created, it is immutable and you can always download it later, after you have changed the questionnaire, and it will still be the same.
List of the documents on the project.#
New document#
We can click on New document when we want to create a new document. We need to give a name to our new document (project name is prefilled) and choose the document template and format. If there is a default document template and format set for the project, they are prefilled in this form. However, we can change them to whatever we want before creating the document. Once we hit Create, we are taken back to the document list and we’ll see the new document there (it might take while before it is generated though).
We can choose any compatible document template and format when creating a new document on the project.#
Document Submission#
Submission services can be used to quickly submit the document directly from DSW to some external service.
Submission service selection for a document.#
If there are any document submission services configured for our DSW instance, we should be able to see Submit option when we open document menu in the document list by clicking on the triple dots. Then we choose the desired from the list and click on Submit.
We can see all the submissions for each document in the document list as well.
Settings#
In the Settings tab, we can configure some details about the project. First we have a name and a description to identify the project.
Next, we have Project Tags. These can be used for providing some metadata or categorization of the project. In the project list, we can filter the projects by these tags. We can write any text we want as a project tag and DSW will suggest us the tags that are there already in use so we can keep them consistent.
Note
Project tags might be disabled in some DSW instances.
Project settings.#
Default Document Template#
We can set a default document template and a default document format. These are then used in the preview tab and also pre-selected when creating a new document.
Project Template#
We can use the project as a project template. If we enable this option, researchers can use it when creating a new project from project template. Note that we also need to make it visible for other logged-in users in sharing settings.
Note
Project template options are visible only for Data Stewards or Admins.
Knowledge Model#
We can see the knowledge model and its tags used for creating the project. If we want to change that, we can simply create a project migration.
Danger Zone#
In the danger zone, we can delete the project. Once the project is deleted it can no longer be recovered.
Sharing#
We can share project with other DSW users or even external collaborators. We can access all the sharing option by clicking the Share button in the top right.
Project sharing settings.#
There are different roles how we can share the project that can access different project features:
| Viewer | Commenter | Editor | Owner | |
|---|---|---|---|---|
| View project | ✔ | ✔ | ✔ | ✔ |
| Metrics | ✔ | ✔ | ✔ | ✔ |
| Preview | ✔ | ✔ | ✔ | ✔ |
| View documents | ✔ | ✔ | ✔ | ✔ |
| Comment on project | ✔ | ✔ | ✔ | |
| Change answers | ✔ | ✔ | ||
| TODOs | ✔ | ✔ | ||
| Version history | ✔ | ✔ | ||
| Create documents | ✔ | ✔ | ||
| Editor notes | ✔ | ✔ | ||
| Sharing settings | ✔ | |||
| Project settings | ✔ |
Note
Some of the following options might be globally disabled in the application settings for the whole DSW instance, therefore not visible on the project level.
The following video tutorial explains and showcases sharing options and tools that can be used while collaborating with others. Some features mentioned in the video are also explained in the project questionnaire.
Users#
We can choose specific users from the DSW instance and their role on the project to grant them access to project features based on the table above. This is a good way to add other collaborators that work together with us on the project. Also, this is the only way to add other project owners.
Visible by all other logged-in users#
We can enable the Visible by other logged-in users toggle to grant access to the project for all other users without the need to explicitly list them. Then, we can also choose what the users can do – view, comment, or edit the project. We cannot grant owner access this way though.
This can be useful, when we want to have an example project accessible by everybody. We can simply enable this and choose that other users can view the project. We also want to set this up when we create a project template.
Public link#
We can enable the Public link toggle to grant access to the project to anyone who has the link. We can again choose what they can do – view, comment, or edit the project. Also, the public link is visible there, so we can simply copy it and send to whoever we want to collaborate with. Then, they don’t need a DSW account and still be able to access our project.
Project Migration#
Every project is based on a specific knowledge model, its version, and selected tags. Sometimes, we might want to change the knowledge model to a different version (for example, when a new version is released), change the knowledge model (for example, when a new customization is created), or just change the tag selection. Project migration is a process where we can do this.
Creating a Project Migration#
We can start a project migration either from the project list, or from the project settings. Sometimes, when there is a newer version of the knowledge model available, we can see a tag update available next to the project name. We can click on the tag to start the migration as well.
Choosing a new knowledge model for the project when creating a project migration.#
We can see the original knowledge model, its version, and selected question tags on the left side. On the right side we can choose new values for all of these. After we are satisfied with our selection we can click on Create button.
Note that the original project will remain unchanged until the migration is finished. So we can cancel it anytime before it is finsihed without affecting the project.
Project Migration#
The next screen is the project migration itself. We can go through all the changes between the orignal and the new knowledge model that affects our answers. During this process, we can also add todos if we want to come back to a specific question later, after the migration.
Reviewing changes during the project migration.#
It is possible that there are no changes to review. This can happen when we don’t have all the answers in the questionnaire yet and those we have are not affected by the changes, i.e., all of the questions that we answered are in the original and in the new knowledge model.
We can leave the migration at any point now and come back to it later. We will see the project twice in the project list, one of them tagged as migrating. If we open the migrating one, we can come back to the project migration. If we open the other one, we can access the original project, however, only in the read-only mode until the migration is finished or cancelled.
Cancelling a Project Migration#
At any point before we finalize the migration, we can decide that we actually don’t want to do the migration. We can simply navigate to the project list and choose the Cancel migration action on the project tagged as migrating. This will cancel the migration and the original project will remain unaffected.
Finishing a Project Migration#
After we resolve all the changes (or if there are no changes to review), we can click on Finalize migration. This will complete the project migration applying all the knowledge model changes, and unlocking the project from the read-only mode.
Project Templates#
Note
Only the data stewards or admins can create project templates.
When creating a new project, we need to choose a knowledge model and optionally select some question tags. After the project is created, we should also choose a default document template and format to enable preview. It can be overwhelming for new researchers to set up everything when they are new to all this.
Project templates are special type of projects where we can set up everything – choose a knowledge model and question tags, set up default document template, pre-fill some answers, add TODOs, comments or editor notes. Researchers can then pick from these project templates when creating a new project. The new project will be the exact copy of the project template so they don’t have to set those things themselves and they have an easier start to their data management planning.
When we want to turn a project into a project template we need to go to the project settings and check the Project Template checkbox.
External Resources#
Project Importers#
Warning
Project importers are an experimental feature.
We can use project importers to import data from different DSW instances or even different applications to DSW. Each has a set of supported knowledge models defined. This is because each knowledge model has a different structure and the importer needs to understand it so it can import the answers to the correct questions.
Note
Only data stewards or admins can access project importers.
If we navigate to Projects → Importers, we can see the list of all available importers. We can enable or disable them by clicking on the triple dots icon and choosing the appropriate action.
List of project importers where we can enable or disable them.#
More information about how to develop project importers is available on the project importers development page.
Documents#
As admins, we can quickly browse all documents stored in the DSW instance by navigating to Documents from the main menu. It is possible to search for a document by name or sort them using the name or creation timestamp.
Each document has name, format, and certain size (if the generation is finished). It can be directly downloaded or deleted from the list. Moreover, we can quickly navigate to the project from which the document is created.
In case that there is a document that was not generated due to an error, we can check the error message. Also, if there are some documents got stuck in Queued or In Progress status, we should check the deployment (especially of the document worker component).
List of all documents across projects.#
Administration#
Administration section serves as the name suggests to admins with managing the DSW instance. By navigating via Administration item from the main menu, we can manage the things listed below.
Table of Contents
Settings#
This section covers different settings available to admin users. The settings are categorized as listed below.
Note
Do not forget press Save button to save the changes.
Table of Contents
System Settings#
System settings allow us to configure basics about the organization running the DSW instance, how can user register and login, and finally the privacy and support information for the users.
Table of Contents
Organization Settings#
On this page, we can configure Name, Description, and Organization ID for our DSW instance. The organization ID can contain alphanumeric characters and dot symbol (but cannot start or end with dot).
Note
It is recommended to use period as for domain name or to use it for capturing organizational structure in the ID, for example, faculty.university or group.faculty.university.
Another part of settings on this page are Affiliations. It servers to pre-define a list of possible affiliations (on per line) that will be suggested to users while they register or update their profiles.
Authentication Settings#
The Default Role settings option allows us to define which role is assigned to new users (see user roles for details about permissions).
Warning
It is recommended to set this to the lowest role possible, i.e. Researchers. Otherwise, new users will be able to change the content for all other users in the DSW instance.
Internal#
For internal authentication, we can set whether the Registration is enabled or not. If enabled, any user who can visit the DSW instance may sign up (and obtain the default role).
Note
In case we are using OpenID or creating user accounts manually, registrations should be disabled.
Another option is whether the Two-Factor Authentication (2FA) is enabled. If enabled, once users try to log in using credentials, they receive an email message with one-time code to confirm the login. Moreover, we can configure Code Length (how many character the code has) and Expiration period in seconds.
External#
Using these settings we can add OpenID Services to allow logging into the DSW instance via external identity provider. First, press Add and fill ID of the service (use only lowercase alphanumeric characters or dash symbols). Then, we should prepare the client application on the side of OpenID service:
Use Callback URL (and optionally Logout URL) to create the client
Obtain Client ID and Client Secret
Obtain OpenID endpoint URL (we may get one ending with
/.well-known/openid-configuration, if so we just use the part before this suffix)Configure the client to have the following claims:
openid,profile,emailConfigure the client to provide the following details in ID tokens:
email,given_name,family_name
Back in the DSW settings, we can fill Client ID, Client Secret, and URL from our OpenID client together with optional Parameters (usually not needed). Finally, we can configure how the log-in button will look like by setting Icon (by using Font Awesome), Name, Background, and text/icon Color.
Note
After setting a new OpenID service, we should directly test it and verify that the configuration works well. For that, we can simply open our DSW instance in a new anonymous window of the web browser.
Example configuration of OpenID service.#
Privacy & Support Settings#
To request users to agree with Privacy Policy and/or Terms of Service documents, we can configure URLs to locate such documents. Then, when new users register to the service, they will be prompted to agree with the linked documents. Note that usually you should inform already registered users in case you change such documents.
Support#
These settings also allow us to configure Support Email that users can use to request help or report issues. Similarly Support Repository Name and Support Repository URL can be used in case we want users to create tickets in issue tracker of some repository, e.g., on GitHub. These support links are then shown in Report issue modal window.
User Interface Settings#
User Interface (UI) settings allow us to manage how the DSW instance looks like: styling, titles, or the dashboard shown when user logs information.
Table of Contents
Dashboard & Login Screen Settings#
The dashboard settings allows us to adjust what users will see after they log in, i.e. on the application initial page called the dashboard.
Dashboard Style#
We can select the Dashboard Style whether the user should see a standard welcome screen which just greets the user in the application, or a role-based dashboard which contains widgets based on current user’s role (see User Roles):
Researcher
Recent Projects Widget contains a list of recent projects of the user for a quick navigation.
Create Project Widget lets the user quickly start a new project.
Data Steward
Create KM / Project Template Widgets let the user to quickly start a new knowledge model editor or project template.
Outdated KM / Document Templates Widgets allow to quickly see outdated packages and document templates in case the DSW Registry connection is configured.
Import KM / Document Template Widgets allow to proceed easily to import of a knowledge model or a document template in case the DSW Registry connection is configured.
Administrator
Outdated KM / Document Templates Widgets allow to quickly see outdated packages and document templates in case the DSW Registry connection is configured.
Usage Widget summarises the usage just as is also possible to see in the Usage.
Configure Organization Widget quickly navigates to Organization Settings if it is not yet done.
Configure Look and Feel Widget quickly navigates to Look & Feel Settings to adjust style of the DSW instance.
Connect DSW Registry Widget quickly navigates to DSW Registry Settings to configure the connection (if not yet been done).
Add OpenID Widget quickly navigates to Authentication Settings to configure the identity provider services (if not yet been done).
Login Info#
It is possible to write a message that users will see before logging in the DSW instance, using HTML or Markdown. The Login info is placed in the center of the login screen. We can use it to explain users in what cases they can/should use our DSW instance, how they should log in (e.g. if we have more authentication services configured), or if there is any news regarding our DSW instance.
Announcements#
Another option to adjust the dashboard and/or the login screen is to add Announcements. Announcements are displayed above the main content in the login screen. In dashboard, they are also displayed above the main content for both welcome and role-based dashboard style. There are three levels of Announcements:
Info - light blue color for sending informations to the users.
Warning - yellow to warn the users about something.
Critical - red to signalize the Announcement is critical and it needs attention.
The content of the Announcement can be edited using Markdown. There are also two additional switches which determine, where the Announcement is displayed. The Announcement can be set up to be displayed either on the dashboard after users log in or on the login screen before the users log in. It is also possible to display the same Announcement in both places. Number of Announcements is not limited.
Look & Feel Settings#
This part of settings allows us to adjust how the DSW instance looks like.
Application Titles#
There are two titles that we can set. First, Application Title is the full title that should identify the DSW instance, for example, in browser’s tab. Second is Short Application Title which is visibile, for example, at the top of the main (left) menu next to the icon. We should keep Short Application Title really short (about 10 characters at maximum) so it fits well.
Logo and Colors#
If enabled by deployment, we may also change the Logo and set Primary Color and Illustrations Color to adjust the visual style to branding of our organisation or basically to our taste.
Note
As this requires re-compilation of stylesheets, the deployment must be adjusted to enable this options.
Custom Menu Links#
We can easily add custom links to the main (left) menu by clicking Add under Custom Menu Links. For each link, we can set Icon (from Font Awesome), Title and the target URL. We can also set whether the link should open in New Window (if not, it will nagivate user directly in the same window/tab from DSW instance). Once the links are there, we can manage them or delete them at this place.
Example configuration of a custom menu link.#
Content Settings#
This part of settings allows us to configure various content-related things such as the connection with DSW Registry for easy imports, Knowledge Models, Projects, and Document Submissions as listed below.
Table of Contents
DSW Registry Settings#
In this settings, we can configure a connection to DSW Registry that will allow import of various content (Knowledge Models, Document Templates, and Locales) to our DSW instance.
Upon enabling the DSW Registry option, we are prompted to enter a Token. It can be obtained either by direct registration in the DSW Registry<https://registry.ds-wizard.org> or through clicking Sign up button. After clicking the button, we will need to enter details about organization (it is prefilled from Organization Settings) and email address to which the confirmation will be sent (prefilled by email of the current user). Then, after clicking a link in the confirmation email, the token will be prefilled automatically. After having the token filled in either way, we can Save the settings.
After successfully setting the DSW Registry, we will see option to import from it for Knowledge Models, Document Templates, and Locales.
Note
The Token value is encrypted in the database.
Knowledge Models Settings#
Public Knowledge Models#
If we want to let users to see and browse certain Knowledge Models (specifically, visit the KM detail and the KM preview) even if not logged in, we can enable Public Knowledge Models. Then, we need to specify Allowed Packages, e.g. which ranges of verions of a certain knowledge model will be publicly available. A blank value serves as any value, for example, if we fill the Organization ID and Knowledge Model ID but leave Min Version and Max Version, it will result in all version of that knowledge model to be public.
Integration Config#
The integrations specified in Knowledge Models can use configuration values (typicaly secrets such as API keys or tokens) from YAML configuration specified in integration.yml file or the content specified here under Integration Config. The value here can be for example:
dbase:
apiKey: topSecretDBaseApiKey
apiUrl: https://api.dbase.example:10666
Note
This configuration value is encrypted in the database.
Projects Settings#
Project Visibility#
If we want to let users select visibility of their projects within the DSW instance, we can enable Project Visibility feature. If it is disabled, the new projects will have the Default Project Visibility which is used when creating a new project:
Private = the project is visible only to the users with explicit access to the project.
Visible - View = the project is visible in view-only mode to all logged-in users, i.e. all users will be able to see the project in their projects list and access it (but not edit or comment anything unless they are invited with different permissions).
Visible - Comment = the project is visible in comment mode to all logged-in users, i.e. all users will be able to see the project in their projects list, access it and comment it (but not edit anything unless they are invited with different permissions).
Visible - Edit = the project is visible in edit mode to all logged-in users, i.e. all users will be able to see the project in their projects list, access it, comment it, and also edit it (e.g. answer questions or editor notes).
Project Sharing#
If we want to let users select sharing option of their projects within the DSW instance, we can enable Project Sharing feature. If it is disabled, the new projects will have the Default Project Sharing which is used when creating a new project:
Restricted = only logged-in users can access the project depending on the project visibility (no public access for anonymous users).
View with the link = anyone with the link to the project may open it in view mode and browse it.
Comment with the link = anyone with the link to the project may open it in comment mode, i.e. browse it and comment on questions.
Edit with the link = anyone with the link to the project may open it in edit mode, i.e. browse it, comment on questions, and also edit it (e.g. answer questions or editor notes).
Anonymous Projects#
If we have enabled Public Knowledge Models, we can also allow anonymous users to create projects where they will be able to fill questionnaires by enabling Anonymous Projects. These anonymous project then work as any other projects with public link set to edit permissions. However, if a logged-in user accesses such a project, then such a user may claim the ownership by clicking Add to my projects button. Anonymous users cannot create new documents, for that they must register and open the project as a logged-in user.
Feedback#
In case we want to allow users to provide feedback specific to questions directly from questionnaires, we can enable Feedback and configure a GitHub repository which will be used as an issue tracker. We should have a bot/service account created with access to the GitHub repository and obtain Personal Access Token. This account will be used to create the GitHub issues in the repository. Then, we need to simply fill GitHub Repository Owner, GitHub Repository Name, and the Access Token.
Note
The Access Token value is encrypted in the database.
Project Tagging#
If enabled, users will be able to tag their projects (using so-called Project Tags) and then use those tags to filter the projects list. The users will be always able to write their own tags but we can provide a list of pre-defined Default Project Tags (one per line).
Document Submission Settings#
If we enable the Document Submission feature, users will be able to see Submit option for their documents. After selecting it, they will be prompted to select a service compatible with the document where they want to submit it.
Each service must have its own ID (recommended is to use lowecase alphanumeric symbols and dash symbols). Then, we can set human-readable Name and Description (Markdown-enabled) to clarify for users what is this service and in what cases they should use it. We also need to specify the Supported Formats, for each we need to select a document template, its version, and the desired format. Finally, we configure how the document is sent to the external service, the request may contain some User Properties (users will be able to set values for them in their user profiles) and it is a HTTP request with a specific Method, sent to the URL, possibly with HTTP Headers. The very last option is to check whether the file should be sent as Multipart (with its own File Name) instead of plainly in the request body. Most of this configuration should be specified by the external submission service.
Note
In case we will update the document template used in Supported Formats, we should verify that it is still suitable for the submission service and if yes, then add it as a new entry under Supported Formats.
Info#
The info section of settings simply serves to let us check various information and statistics about the DSW instance.
Table of Contents
Usage#
Usage allows us quickly see numbers of entities in the DSW instance such as number of users, active users, knowledge models, KM editors, templates, projects, or documents. Moverover, we can see storage usage, i.e. how much capacity is being used by documents and templates (their files). We cannot perform any actions on this page.
Usage statistics.#
Users#
Users list allows administrators to see and manage all users in a DSW instance. The list can be filtered using role, searched based using name or email fragment, and sorted via various properties of users. The list shows the role of a user next to its name and also indicates in case the user is inactive. Next to the email, we can quickly see what authentication services the user uses to log-in (internal is the internal authentication with email-password credentials, other are based on configured OpenID services).
A user detail can be opened by clicking the name of a user or by selecting Edit in the right dropdown menu for the desired row. There, a user can be also deleted via the Delete action. Finally, administrator can create a new user by clicking Create.
List of users.#
Table of Contents
Create User#
As administrators, we can create new users manually by clicking Create on the users list and submitting the form. Each user must have a unique email address, first name and last name, assigned role, and a password. Optionally, a user can have affiliation specified.
Note
If the user is created by administrator, the user is activated by default and no email is sent to the user.
User Detail#
As administrators, we can edit existing users manually on the detail (selected user from the users list). It is possible to change all properties of the user, including possibility to change whether the user account is active or inactive.
Detail of a user profile.#
The password can be also changed (after selecting Password from the left navigation of user settings).
Form for changing password of a user.#
User Roles#
There are three user roles available: researchers, data steward, and admin. Permissions are associated with the roles, basically they affect what the users can do in a DSW instance:
| Researcher | Data Steward | Admin | |
|---|---|---|---|
| Projects | ✔ | ✔ | ✔ (all) |
| Knowledge Models | ✔ (read-only) | ✔ | ✔ |
| Knowledge Models Editors | ✔ | ✔ | |
| Document Templates | ✔ | ✔ | |
| Settings | ✔ | ||
| Locales | ✔ | ||
| Users | ✔ |
Locales#
After navigating to Locales (under Administration), we can browse and manage a list of locales in the DSW instance. Similarly to knowledge models and document templates, each locale has its unique identifier and version. Moreover, each locale has a language code specified. In the list we see the latest version and can quickly navigate to Locale Detail (which includes also older versions) by clicking the locale name or selecting View detail from the right item menu of the desired row.
There is always the Engligh locale (wizard:default:1.0.0) which is embedded and cannot be deleted. For others, we can use Export and Delete options from the right item menu.
Another option is to switch other locale to be the default one using Set default action. The default locale will be used in case no available locale tgat matching user’s preferences (explicit or implicit from the web browser). We can Disable or Enable locales except the default one (which must be enabled).
If there is a locale with newer version available in the DSW Registry (and if configured), update available clickable badge may appear. Finally, we can use Import to Import Locale and Create to Create Locale.
Note
We support community of DSW translators by managing the repository ds-wizard/wizard-client-locales and service for translating using web browser localize.ds-wizard.org.
List of locales.#
Table of Contents
Import Locale#
We can import an existing locale by navigating to in the main menu and then clicking on Import button on the list of locales.
From DSW Registry#
If the DSW instance is connected to the DSW Registry, it is possible to import locales from it by entering the locale ID of desired template (e.g. dsw:cs:0.2.0) and pressing the Import button.
Note
In case of locales present in the DSW Registry, we will be notified about the available upgrades.
Input for importing a locale from DSW Registry.#
From file#
We can import a locale as a ZIP package. Such a package can be created as an export from DSW.
Input for importing a locale using a ZIP package.#
Create Locale#
We can create a new locale directly in DSW by pressing Create from Locales. We need to fill the details about the new locale such as name, description, language code (RFC5646, e.g. en or en-GB), locale ID, locale version, license, README (with Markdown syntax), and recommended app version. The recommended app version captures for which version of the DSW is this locale intended and compatible with (it can be used in other versions as well but may have some untranslated texts).
Finally, a PO file is requested from us. We can create such PO file in a standard (gettext-based) way. The needed POT file is always part of the release attachments of the DSW client application (select the desired release and there is wizard.pot asset).
Locale Detail#
The detail of a locale provides us information about a locale after navigating to it from Locales. The detail shows basic information about the locale such as its name, ID, language code (RFC5646, e.g. en or en-GB), version, recommended compatible DSW version, license, and indication whether the locale is enabled or not.
The main part of the detail is the README of the locale that should contain basic information and changelog. In the right panel under the basic information, we can navigate to other versions of the locale.
In the top bar, we can Export the locale as a ZIP package, set or unset it as default, Enable or Disable it, or Delete this version of the locale.
Note
The default locale must be always enabled as it serves for users that did not have any preference or do not request matching locale directly using the browser configuration.
Detail of a locale.#
Profile#
As every logged-in user, we can manage our own profile. We can see the options by hovering over the box with our name and avatar in the lower part of the left sidebar with main menu (only avatar is shown in case of collapsed sidebar). From there, we can navigate to Edit Profile. If locales are configured, we can also Change Language.
Moreover, there are also options to Log out, see basic information about the DSW instance using About, or in case of problems we can use Report issue to know how to proceed.
Profile menu.#
Table of Contents
Edit Profile#
After navigating to Edit profile from the Profile menu, we can change the information of our profile: Email address, First name, Last name, and Affiliation. We need to submit the changes by clicking Save button.
In case of configured submission services, there might be additional inputs under Submission Settings such as API tokens or other information used for the document submission.
Form for editting profile with example submission settings.#
Note
The values of Submission Settings are treated as potentially sensitive information; thus are stored encrypted.
If we want to Change Password, we need to switch to Password from the left menu titled User settings.
Table of Contents
Change Password#
The password be can changed after navigating using Password from the Edit Profile. Here we can simply enter new password (it must be strong enough), repeat it again and press Save button.
Form for changing password.#
API Keys#
When we want to access the DSW through DSW API we have to set up an API Key. The API Key has an API Key Name so we can remember for what purpose this Key is used and Expiration which is a date from when the API Key will no longer be valid.
Form for creating an API Key.#
After filling out the API Key Name and Expiration, an API Key is generated. In this step we have to copy the API Key, because after seeing it once, it is no longer possible to access it again.
After we click on Done button, the new API Key is hidden and the information about this key is added to the table below, that contains all Active API Keys.
Active Sessions#
Here we can see list of active sessions that are logged into our account. In case we don’t longer want any of them to have access to our instance, we can revoke the session.
List of active sessions.#
Change Language#
A user can explicitly select a desired language after clicking Change language from the Profile menu. In case the language becomes unavailable later after the selection, it will fall back to the one marked as default.
Modal window with language selection.#
Note
The selection of language is saved only locally in the browser (in local storage), so if we log in from different locations, we need to select the language there again.
Roadmap#
Need to know some info about the latest released version or what is planned?
Note
The backlog is being highly affected by provided feedback, community needs, and discussion with our key users. You can visit our Ideas and Jira or check out DSW Advisory Board.
Warning
You should always strive to use the latest version of DSW for both security and user experience reasons. Also, if you are having a self-hosted DSW, please check Upgrade Guidelines prior to actual upgrade.
Planned Versions#
3.24#
Release (expected): 30 May 2023
Released Versions#
3.23#
Release: 2 May 2023
Features:
Added the possibility to generate API keys to access the API instead of using username and password. The API keys also work when 2FA is enabled.
Added an overview of all active sessions.
It is now possible to use HTML for login info.
Added possibility for sidebar login info under the login box.
Welcome warning and info have been reworked to announcements – it is now possible to have an unlimited list of announcements of different levels and choose if they are visible on the dashboard and/or login screen.
Added sort by created to document template list.
Improved progress bar in project migration.
The warnings tab in the knowledge model editor is now automatically closed when the last one is resolved.
Improved form actions to make them more visible when forms change.
Bugfixes:
Fixed project indication calculation after import or project migration.
Fixed double error message when deleting failed in list views.
Fixed buttons in email templates in Outlook.
Fixed phase in a questionnaire after project migration if the phase no longer exists.
Fixed dropdown menus in the sidebar when the page was scrolled.
Fixed knowledge model export from the knowledge model list.
Misc:
Changed the path of configuration files (see upgrade guidelines).
Sped up processing and generating of documents.
More:
3.22.1 (tools)#
Release: 14 April 2023
Bugfixes:
Fixed sending mails when configuration is loaded from database.
More:
3.22.3 (backend)#
Release: 13 April 2023
Bugfixes:
Fixed the selected phase in projects when migrating from a knowledge model without phases to a knowledge model with phases.
More:
3.22.2 (backend)#
Release: 12 April 2023
Bugfixes:
Fixed an issue that sometimes caused suggesting the same knowledge model multiple times when creating a new project or knowledge model editor.
More:
3.22.1 (frontend, backend)#
Release: 11 April 2023
Bugfixes:
Fixed database migration of existing KM editors after 3.22 that could cause unexpected KM editor version or missing metadata (such as readme).
Fixed publish process in KM editor and Document Template Editor that could be confusing after 3.22 changes.
Fixed deleting KM editor when it is migrating.
More:
3.22.0#
Release: 4 April 2023
Features:
Added the possibility to set a knowledge model as deprecated so researchers cannot use it to create new projects.
Added phase editor to KM Editor (similar to Tag editor).
Renamed Template tab to Settings in the document template editor to make it consistent with KM Editor or Project.
Added link to selected project in document template editor preview.
Position in the questionnaire is now remembered when switching tabs in the project (such as going to preview and back to the questionnaire).
Warnings tab in the project is now automatically closed when the last one is resolved.
Projects are no longer filtered by current user if the user is admin.
Improved accessibility of unanswered question indications and metrics (as well as adding an option to hide non-desirable questions).
Added information about a version of all components in the About modal.
Improved add button labels in various forms to make it easier to understand what they add.
Added support for DKIM signing for emails.
Added experimental weasyprint step in document templates for better PDF documents generation.
User details are now updated in the menu after editing your own profile.
Added link to the DSW Registry from locale detail.
Bugfixes:
Fixed visible first chapter in KM Editor preview when deleted.
Fixed inconsistent update label for badge and action for KM migration.
Fixed failing to publish knowledge models due to wrong event squashing in some cases.
Fixed redirect to login when opening the project after the session has expired.
Fixed a visual bug in the project selection dropdown in the document template editor preview.
Fixed text overflow for long questions/answers in the project import view.
Fixed image previews in the document template editor.
Fixed downloading document template with DSW TDK.
Fixed dropdown menu separators in list views.
Misc:
Added support for RO-Crates (RO-Crate Importer and RO-Crate Template)
Improved default English locale metadata.
Added support for arm64 builds for most of the Docker images.
More:
3.21#
Release: 7 March 2023
- Key changes:
Two-factor authentication (2FA)
i18n support in document templates
RO-Crate import/export
Warnings on imports
Various optimizations and UI fixes
3.20#
Release: 7 February 2023
- Key changes:
Document template editor (idea)
Mark document template as legacy
Various UI improvements and fixes
3.19#
Release: 3 January 2023
- Key changes:
Indications computation
Minor UI improvements and fixes
3.18#
Release: 29 November 2022
- Key changes:
Localizations (idea)
Filter file extensions when importing KM or template
Logout user when 401 received from API on dashboard
3.17#
Release: 1 November 2022
- Hotfixes:
3.17.1 (frontend), 14 November 2022, Jira
3.16#
Release: 4 October 2022
- Key changes:
Import for replies from other questionnaires (idea)
Collapsible and movable items in list questions
Main menu grouping
Speed optimizations and refactoring
3.15#
Release: 5 September 2022
- Key changes:
Project loading optimization
Python components refactoring
Several other fixes and refactoring
3.14#
Release: 2 August 2022
- Key changes:
Migrate to Bootstrap 5
Improve authentication for downloads
Python components refactoring
3.13#
Release: 28 June 2022
- Key changes:
Prevent user leave unsaved changes
Improved exceptions monitoring
3.12#
Release: 31 May 2022
- Key changes:
New types of value questions
KM events optimizations
Several bugfixes and UI/UX improvements
3.11#
Release: 3 May 2022
- Key changes:
Apply all action for KM migrations
Improved efficiency of document worker
Auto-upgrade default document templates in project
Several bugfixes and UI improvements
3.10#
Release: 5 April 2022
- Key changes:
Mailer
Integration widget
Opening Markdown links in new tab/window
Several bugfixes and UI improvements
3.9#
Release: 1 March 2022
- Key changes:
Basic password requirements
KM Editor: list of questions used with integration
Improved project migration
Usage statistics for administrators
Several bugfixes and UI improvements
- Hotfixes:
3.9.1 (wizard-server), 8 March 2022, Jira
3.8#
Release: 1 February 2022
- Key changes:
Online collaboration in KM Editor
3.7#
Release: 4 January 2022
- Key changes:
Projects tagging and filtering
3.6#
Release: 7 December 2021
- Key changes:
Enhancing integration question options (item template)
- Hotfixes:
3.6.1 (document-worker), 9 December 2021, Jira
3.5#
Release: 2 November 2021
- Key changes:
Additional metadata for KM entities
Improved document submissions
Admin operations
3.4#
Release: 5 October 2021
- Key changes:
Comments in projects
New Jinja filters for document context handling
3.3#
Release: 8 September 2021
- Key changes:
Improved default document template
Improved template development experience
Enhanced Search API
Several fixes
3.2#
Release: 3 August 2021
- Key changes:
Custom metrics (in KM)
Custom phases (in KM)
Several optimizations
3.1#
Release: 25 June 2021
- Key changes:
Project templates
Minor UI improvements
3.0#
Release: 1 June 2021
- Key changes:
Migration from MongoDB and RabbitMQ to PostgreSQL and S3
Deep links feature
2.14#
Release: 4 May 2021
- Key changes:
Submitting forms using Enter key
Shortcuts for KM Editor and Forking KM
Clarified public link for project in UI
2.13#
End of development: 31 March 2021
Release: 7 April 2021
- Key changes:
Auto-reconnect in questionnaires (websockets)
Fix text inputs in questionnaires when using Grammarly in browser
Added actions directly to list views of knowledge models and templates
2.12#
End of development: 2 March 2021
Release: 12 March 2021
- Key changes:
Questionnaire versioning (Version History)
2.11#
End of development: February 2021
Release: February 2021
- Key changes:
Add multiple choice question
Show tags in the questionnaire
2.10#
End of development: January 2021
Release: January 2021
Key changes:
Possibility to add specific users to the questionnaire as collaborators
2.9#
End of development: 30 November 2020
Release: 9 December 2020
- Key changes:
Refactored error messages
Several bugfixes
2.8#
End of development: 27 October 2020
Release: 3 November 2020
- Key changes:
Pagination & sorting in table views
Introduced DSW Template Development Kit
Minor UX improvements
- Hotfixes:
2.8.1 (wizard-server), 24 November 2020, Jira issues 2.8.1
2.7#
End of development: 29 September 2020
Release: 5 October 2020
- Key changes:
Improved caching for speed optimization
Reworked questionnaire detail
2.6#
End of development: 5 September 2020
Release: 9 September 2020
- Key changes:
Added questionnaire live collaboration
Introduced Projects to relate questionnaire, TODOs, documents, and settings
Several UI/UX improvements
Improved design of email templates
2.5#
End of development: 24 June 2020
Release: 8 July 2020
- Key changes:
Added templates management
Several UI/UX improvements
Introduced backend workers for scheduled/async tasks
Added option to disable questionnaire summary report
2.4#
End of development: 27 May 2020
Release: 3 June 2020
- Key changes:
Added RDF support step in document worker
Improved default naming of new documents
Minor UI/UX improvements
Several bugfixes
2.3#
End of development: 29 April 2020
Release: 6 May 2020
- Key changes:
Enhanced backend logging for ELK
Added document submission
Improved integration with Registry for simpler Sign Up
Added user avatars
Several bugfixes and optimizations
2.2#
End of development: 25 March 2020
Release: 1 April 2020
- Key changes:
Added support for OpenID
Added affiliations in user profiles
Introduced settings to change configurations directly in DSW interface
Added API documentation using Swagger
UI/UX improvements
Several bugfixes and optimizations
2.1#
End of development: 25 February 2020
Release: 3 March 2020
- Key changes:
Introduced document worker for better scalability
Migrated backend to new framework
Added dropdown actions to list views
Several bugfixes
2.0#
End of development: 14 January 2020
Release: 14 January 2020
- Key changes:
Added move functionality for knowledge models
Added possibility to assign template to KMs
Added questionnaire cloning
Added expand/collapse all in KM Editor
Internal refactoring and structure enhancements
Several bugfixes
1.10#
End of development: 27 August 2019
Release: 3 September 2019
- Hotfixes:
1.10.1 (wizard-client), 18 September 2019, Jira issues 1.10.1
1.9#
End of development: 23 June 2019
Release: 30 June 2019
- Hotfixes:
1.9.1 (wizard-server), 7 August 2019, Jira issues 1.9.1
1.9.2 (wizard-server), 13 August 2019, Jira issues 1.9.2
1.8#
End of development: 11 June 2019
Release: 13 June 2019
- Hotfixes:
1.8.1 (wizard-client), 13 June 2019, Jira issues 1.8.1
1.7#
End of development: 15 May 2019
Release: 16 May 2019
1.6#
End of development: 30 April 2019
Release: 7 May 2019
1.5#
End of development: 2 April 2019
Release: 9 April 2019
1.4#
End of development: 3 March 2019
Release: 10 March 2019
1.3#
End of development: 3 February 2019
Release: 10 February 2019
1.2#
End of development: 6 January 2019
Release: 13 January 2019
- Hotfixes:
1.2.1 (wizard-server), 14 January 2019, Jira issue 1.2.1
1.1#
End of development: 9 December 2018
Release: 16 December 2018
1.0#
End of development: 24 October 2018
Release: 30 October 2018
Release Management#
Currently, we release in a montly cycle (first Tuesday in month). We strictly use semantic versioning. In case of urgent bug reported or vulnerability identified, we release a hotfix (patch) for the latest major and minor version. The major and minor version must be matching for compatibility reasons; however, patch version might differ.
We release multiple components from the same repositories:
frontend =
wizard-client,registry-client,wizard-style-builderbackend =
wizard-server,registry-servertools =
mailer,document-worker,tdk,data-seeder(and other Python libraries)
When a hotfix is released, it is always for all the components in the affected repository.
Self-Hosted DSW#
This section explains how to deploy, configure, and maintain DSW instance on your own.
Note
Before going for self-hosted options, it is a good idea to try out DSW using the already-configured publicly-available instances.
Table of Contents
Deployment#
Own DSW Instance#
Warning
For production use, we should consult deployment and other operations with the sysadmin or contact the DSW Team for professional services.
The following instructions are intended only for local deployment and trying it out. It does not include any safety measures that are required for production deployment and are handled differently based on specific requirements and deployment sites (e.g., using HTTPS, enabled websockets, backups, metrics-gathering, and many others).
Deployment with Docker#
The simplest way is to use Docker Compose. Requirements are just to have Docker installed, privileges for current users, and the Docker daemon started. At least basic knowledge of Docker is required.
Clone the DSW Deployment Example repository.
Check config files (described in Configuration), especially
dsw.ymlanddocker-compose.ymlRun the DSW with Docker compose
docker-compose up -dAfter starting up, we will be able to open the Wizard in our browser on http://localhost:8080
We can use
docker-compose logsto see the logs anddocker-compose downto stop all the services
Warning
It is important to use latest patch version, we should monitor the documentation or join the community Slack if new hotfix releases are available.
Deployment without Docker#
It is highly recommended using Docker. However, we can compile and run everything directly on our machine. Nevertheless, an additional expertise in programming and building/running Elm, Haskell, Python, and others is required.
The related code and instructions are available:
Default Users#
Initially, migrations will fill the database with predefined data needed including three users, all with password - password:
albert.einstein@example.com (Administrator)
nikola.tesla@example.com (Data Steward)
isaac.newton@example.com (Researcher)
We can use those accounts for testing or initially make our own admin account and then delete them.
Warning
Having a public instance with default accounts is a security risk. We should delete or change default accounts (mainly Albert Einstein) if our DSW instance is public as soon as possible.
DSW Registry#
When we have our own self-hosted instance, it is essential to register within the DSW Registry. It is a source of shared knowledge models, document templates, and locales that can support our deployment. The registry is also integrated inside the DSW. Therefore, we can easily pull new versions from the DSW. The registration can be done either directly in our DSW instance in Settings or via the DSW Registry website.
Initial Knowledge Model, Document Templates, and Locales#
When we have a fresh installation, there are just the default users and no knowledge models or document templates. We are free to create them from scratch if we want. Another option is to import existing KM and document templates from the DSW Registry.
In the DSW Registry, we can find the core knowledge model for general data stewardship and a couple of document templates, such as Horizon Europe and locales! Another option is to import it from a file if we have any (according to usage).
Database Backups#
If we want to regularly backup our database (and we should!), all we need to do is to set up a cronjob that backups PostgreSQL database (e.g., using pg_dump utility) as well S3 storage.
Deployment Requirements#
The following requirements were estimated using limiting Docker resources provided to containers.
Component |
Minimal |
Recommended |
|---|---|---|
Server |
128 MB |
512 MB |
Client |
16 MB |
64 MB |
Document Worker |
240 MB |
1024 MB |
Mailer |
240 MB |
448 MB |
Total |
624 MB |
2048 MB |
As for the CPU, there are no long-running tasks that would require excessive CPU consumption. Limiting CPU resources can only make some operations slightly longer (e.g., importing a knowledge model or generating a document). A number of CPUs/cores will then affect performance for concurrent users/actions. Memory consumption is affected by the size of the content (as some content is being cached for speed optimizations).
Memory used by document workers might be affected by the size of the document template (and assets) for generating a document. Recommended memory in the table above is approximated for a long-term run (without restarts) with a significant amount of content. It also minimizes the need for garbage collection techniques that may slow down the server component.
Note
Real requirements should be aligned with the intended use (number of concurrent users, number of users in total, size of document templates, etc.). The minimal requirements are sufficient for single-user deployment and recommended requirements should handle tens of concurrent users.
Configuration#
Settings#
Most of the configuration is done through Settings (accessible by Administrator).
Configuration Files#
Configuration files are used for setting the server-side configuration. Since the 3.0 release, the configuration for server and document worker components has overlapped. Therefore, we can have a single configuration file for both.
Server Configuration#
For reference, see the configuration of the DSW Deployment example.
General#
This configuration section is used only by Server and covers basic configuration of the application.
- serverPort#
- Type:
Int
- Default:
3000Port that will be the web server listening on.
- clientUrl#
- Type:
URI
Address of client application (e.g. https://localhost:8080).
- secret#
- Type:
String
Secret string of 32 characters for encrypting configuration in the database.
- rsaPrivateKey#
- Type:
String
RSA private key for signing JWT tokens according to RS256 algorithm in PEM format (e.g. use
ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.keywithout passphrase and paste the contents to this configuration item).
Warning
We should keep our secret and rsaPrivateKey secured! Changing secret will require re-configuration of secrets stored in the database, e.g., token for Registry.
If we need to change our secret, we need also replace all values encrypted by the secret that is stored in the database as follows:
Note somewhere values from Settings: Client ID and Client Secret of OpenID configurations, Registry token, and GitHub token for Feedback functionality, etc. Adjust the settings that the values are not there (recommended; e.g., remove OpenID configuration), and save it.
Change the
secretin the configuration file and restart the DSW server (re-create the container if using Docker).Adjust the settings back to our previous values.
If we also use some “user properties” (for the Document Submission feature), let our users know to change the values in their profiles.
Database#
Information for connection to PostgreSQL database.
- database.connectionString#
- Type:
String
PostgreSQL database connection string (typically:
postgresql://{username}:{password}@{hostname}:{port}/{dbname}, for example,postgresql://postgres:postgres@localhost:5432/postgres).
S3#
Information for connection to S3 storage (used for document and document template assets).
- s3.url#
- Type:
URI
Endpoint of S3 storage, e.g.,
http://minio:9000
- s3.username#
- Noindex:
- Type:
String
Username (or
Access Key ID) for authentication
- s3.password#
- Type:
String
Password (or
Secret Access Key) for authentication
- s3.bucket#
- Type:
String
- Default:
engine-wizardBucket name used by DSW
Warning
S3 service must be publicly accessible (so users can download documents and export templates or locales). Also, bucket must be created otherwise documents cannot be created and documet tempates / locales imported.
Mail#
This configuration section is used only by Mailer. It must be filled with SMTP connection information to allow sending emails (registration verification, password recovery, project invitation, etc.).
- mail.enabled#
- Type:
String
It should be set to
trueunless used for local testing only.
- mail.name#
- Type:
String
Name of the DS Wizard instance that will be used as “senders name” in email headers.
- mail.email#
- Type:
String
Email address from which the emails will be sent.
- mail.host#
- Type:
String
Hostname or IP address of SMTP server.
- mail.port#
- Type:
Int
Port that is used for SMTP on the server (usually
25for plain or465for SSL).
- mail.ssl#
- Type:
Boolean
- Default:
falseIf SMTP connection is encrypted via SSL (we highly recommend this).
- mail.authEnabled#
- Type:
Boolean
If authentication using username and password should be used for SMTP.
- mail.username#
- Type:
String
Username for the SMTP connection.
- mail.password#
- Type:
String
Password for the SMTP connection.
Externals#
This configuration section is used only by Document Worker. We can affect steps for templates that use external tools (pandoc and wkhtmltopdf). It is usually sufficient to keep the defaults. Each of them has configuration options:
- executable#
- Type:
String
Command or path to run the external tool.
- args#
- Type:
String
Command line arguments used to run the tool.
- timeout#
- Type:
Int
Optional for limiting time given to run the tool.
Integrations Configuration#
Integrations in the DS Wizard use external APIs. Sometimes, we might need some configured variables, such as API keys or endpoints. For example, integration with ID dbase might use the following configuration.
dbase:
apiKey: topSecretDBaseApiKey
apiUrl: https://api.dbase.example:10666
someConfig: someValue4Integration
There can be multiple integrations configured in a single file. These can be used then when setting up the integration in the Editor as ${apiKey}, ${apiUrl}, etc. More about integrations can be found in separate integration questions documentation.
Note
Different knowledge models may use different variable naming. Please read the information in README to find out what is required. We recommend authors to stick with apiKey and apiUrl variables as our convention.
Client Configuration#
If we are running the client app using “With Docker”, the all we need is to specify API_URL environment variable inside docker-compose.yml. In case we want to run the client locally, we need to create a config.js file in the project root:
window.dsw = {
apiUrl: 'http://localhost:3000'
}
The client also provides a wide variety of style customizations using SASS variables or message localization. Then we can mount it as volumes in case Docker as well:
volumes:
# mount SCSS file
- /path/to/extra.scss:/src/scss/customizations/_extra.scss
- /path/to/overrides.scss:/src/scss/customizations/_overrides.scss
- /path/to/variables.scss:/src/scss/customizations/_variables.scss
- /path/to/provisioning.json:/configuration/provisioning.json:ro
# mount other assets, we can then refere them from scss using '/assets/...'
- /path/to/assets:/usr/share/nginx/html/assets
_extra.scss= This file is loaded before all other styles. We can use it, for example, to define new styles or import fonts._overrides.scss= This file is loaded after all other styles. We can use it to override existing styles._variables.scss= A lot of values related to styles are defined as variables. The easiest way to customize the style is to define new values for these variables using this file.
For more information about variables and assets, visit Theming Bootstrap. The color of illustrations can be adjusted using $illustrations-color variable.
Document Templates#
We can freely customize and style templates of documents (DMPs). HTML and CSS knowledge is required, and for doing more complex templates that use some conditions, loops, or macros, knowledge of Jinja templating language (pure Python implementation) is useful. For more information, please read the following section.
Email Templates#
Similarly to document templates, we can customize templates for emails sent by the Wizard located in templates/mail folder. It also uses Jinja templating language. And we can create HTML template, Plain Text template, add attachments, and add inline images (which can be used inside the HTML using Content-ID equal to the filename).
Templates Structure#
The structure is following:
templates/mail/_common= layout, styles, common filestemplates/mail/_common/attachments= attachments for all emailstemplates/mail/_common/images= inline images for all emailstemplates/mail/<name>= templates specific for this email type, should contain message.html.j2 and message.txt.j2 files (or at least one of them, mail servers prefer both variants)templates/mail/<name>/attachments= attachments specific for email typetemplates/mail/<name>/images= inline images specific for email type
All attachments are loaded from the template-specific and common folders and included in to email with the detected MIME type. It similarly works for inline images, but those are not displayed as attachments, just as related part to the HTML part (if present). We highly recommend using ASCII-only names without whitespaces and with standard extensions. Also, sending a minimum amount of data via email is suggested.
Templates variables#
All templates are provided also with variables:
appTitle= from the configurationappTitleclientAddress= from the configurationclientUrlmailName= from the configurationnameuser= user (subject of an email), structure with attributes accessible via . (dot, e.g.user.name)
Email types#
Currently, there are following types of mail:
registrationConfirmation= email sent to user after registration to verify email address, containsactivationLinkvariableregistrationCreatedAnalytics= email sent to address specified in the configuration about registration of a new user (see Analytics config)resetPassword= email sent to user when requests resetting a password, containsresetLinkvariabletwoFactorAuth= email sent to user when the 2FA is enabled
Docker deployment#
Including our own email templates while using dockerized Wizard is practically the same as for DMP templates. We can also bind whole templates/mail folders (or even templates if we want to change both):
mailer:
image: datastewardshipwizard/mailer
restart: always
volumes:
- /dsw/application.yml:/app/config.yml:ro
- /dsw/templates/mail:/app/templates:ro
# ... (continued)
Upgrade Guidelines#
Upgrading DSW#
Warning
Backup database and other important data (e.g., configuration) before upgrade!
Using Docker#
In case of using Docker, just use the tag in docker-compose.yml or pull the new Docker image and restart using down/up:
$ docker pull datastewardshipwizard/wizard-server
$ docker pull datastewardshipwizard/wizard-client
$ docker pull datastewardshipwizard/document-worker
$ docker pull datastewardshipwizard/mailer
$ docker-compose down
$ docker-compose up -d
Other setup (using Git)#
All we need to do is download or checkout the new version from GitHub repositories and rebuild the application (according to the guidelines above):
$ git checkout vX.Y.Z
Upgrade process#
Usually, nothing special is required for the upgrade. Internal structure changes are migrated automatically using DB migrations and Metamodel migrations (since 1.8.0). See below the changes that need to be done by us (since 1.10.0):
3.22.X to 3.23.X#
(breaking) The JWT signing has been changed to RS256 and you need to add
rsaPrivateKeyin configuration file (see Server Configuration).(breaking) The location of configuration files has been changed and unified across components, check the deployment example for details. The main configuration file is located in
/app/config/application.ymlpath which can be adjustedAPPLICATION_CONFIG_PATH.
3.21.X to 3.22.X#
(nothing)
3.20.X to 3.21.X#
(breaking) The
wizard-clientcontainer now exposes a different port (as all images are now root-less): 8080 instance of 80.(breaking) The S3 service must be now publicly available, thus the S3 URL configured via Server Configuration must be reachable by users to support download of documents or document preview.
3.19.X to 3.20.X#
The document template metamodel version is raised to 11. All templates must be updated (changes are only minor in template.json files, see Template Development section for more information).
3.18.X to 3.19.X#
(nothing)
3.17.X to 3.18.X#
(nothing)
3.16.X to 3.17.X#
If we are upgrading from the older version then 3.16.X we need to first upgrade to version 3.16.X.
3.15.X to 3.16.X#
(nothing)
3.14.X to 3.15.X#
(nothing)
3.13.X to 3.14.1#
We need to run wizard-server for the first time while document-worker and mailer are not running (to ensure database migration to proceed). Then we can start everything as usual.
3.13.X to 3.14.X#
(nothing)
3.12.X to 3.13.X#
(nothing)
3.11.X to 3.12.X#
(nothing)
3.10.X to 3.11.X#
(optional) We can now use integration.yaml configuration in Settings instead of the file store on FS and mounted to the Docker container.
3.9.X to 3.10.X#
Standalone mailer component has been introduced. We need to adjust our deployment (e.g., docker-compose.yml) accordingly (see deployment-example).
3.8.X to 3.9.X#
(nothing)
3.7.X to 3.8.X#
All KM migrations must be finished (completed or deleted); otherwise, the upgrade of the backend (database) will fail with the corresponding message in the logs.
3.6.X to 3.7.X#
(nothing)
3.5.X to 3.6.X#
(nothing)
3.4.X to 3.5.X#
The template metamodel version has been updated (to v5). Updating all document templates is needed (annotations were added, so we can safely change version 4 to version 5 without breaking anything).
All KM migrations must be finished (completed or deleted); otherwise, the upgrade of the backend (database) will fail with the corresponding message in the logs.
3.3.X to 3.4.X#
(nothing)
3.2.X to 3.3.X#
(nothing)
3.1.X to 3.2.X#
The template metamodel version has been updated (to v4). Updating all document templates is needed.
All knowledge models have (after the automatic data migration) the default metrics and phases that can be changed in KM Editor.
3.0.X to 3.1.X#
As an administrator, we should either disable the “Project Templates” feature (Settings - Projects - Project Creation, select “Custom only”) or prepare some project templates for our users to avoid confusion.
2.14.X to 3.0.X#
All data must be migrated as we switched from MongoDB and RabbitMQ to PostgreSQL and S3. To support data migration, we provide dsw2to3 tool with step-by-step instructions.
2.13.X to 2.14.X#
(nothing)
2.12.X to 2.13.X#
(nothing)
2.11.X to 2.12.X#
The metamodel for templates has been upgraded, and accessing the reply values is changed due to additional metadata about each reply, see Document Context. But if we are using filters such as
reply_str_value, it gets the reply object with value correctly. Moreover, for working with integration reply, the type values are renamedIntegrationValue->IntegrationTypeandPlainValue->PlainTypefor consistency.
2.10.X to 2.11.X#
If we are using the
questionnaire-reporttemplate, it is recommended to upgrade it to version 1.2.0 (from Registry or GitHub Release) so it displays also new Multi-Choice questions. Otherwise the choices won’t appear in the exported document if there are any.
2.9.X to 2.10.X#
(nothing)
2.8.X to 2.9.X#
(nothing)
2.7.X to 2.8.X#
(nothing)
2.6.X to 2.7.X#
(nothing)
2.5.X to 2.6.X#
The document templates including the default
questionnaire-reportmust be updated from https://registry.ds-wizard.org/templates.Upgraded template metamodel version 2 requires manual migration of custom templates:
questionnaireRepliesMap (map path:Reply) is no longer present in the context
questionnaireReplies is now map with path:ReplyValue, provided filters (such as
reply_str_value) are adjusted but wherever we usedreply.value.valueit should bereply.valuewith this change.Reply for item question is no longer an integer (number of answers) but a list of UUIDs representing the answers instead of integers. We added
reply_itemsto extract the list from a ReplyValue.
Since 2.6.0, we are using WebSockets (for live collaboration). If we are using a proxy, we need to configure it accordingly. For example, in case of Nginx:
server {
# ...
location / {
# ...
# required for websockets
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
proxy_send_timeout 86400;
}
}
2.4.X to 2.5.X#
Document templates have been moved from FS to database. To simplify the transition for custom templates, we added to the Docker image a script that loads templates from FS to the database via DSW API. But there are several new information that we need to provide in
template.jsonfile:id(instead ofuuid),templateId,organizationId,version(semver),license,readme(Markdown). Theidshould be in formatorganizationId:templateId:version. Please note that this applies only for custom templates, default template can be removed from FS as it is added to the database automatically. The script must be enabled by setting envvarENABLE_TEMPLATE_LOAD `` to ``1andSERVICE_TOKENaccording to the configuration.Cron is no longer needed for the feedback synchronization (environment variables in
docker-compose.yml) as DSW schedules synchronization internally.
2.3.X to 2.4.X#
To unify configuration, document-worker now supports and prefers YAML configuration files.
Local/custom
template.jsonfiles must be updated (renamedallowedKMstoallowedPackages, and several new attributes:descriptionfor template andshortName+colorfor each format).
2.2.X to 2.3.X#
(nothing)
2.1.X to 2.2.X#
Configuration of client and several features is now moved from
application.ymlfile to in-app Settings; therefore, it must be reconfigured during upgrade process. Additionalsecretmust be configured inapplication.ymlfor encryption and JWT tokens (JWT.secret section has been removed), see Server Configuration configuration. It is recommended to first add general.secret (32 chars secret), start DSW, migrate options fromapplication.ymlto Settings and then optionally clean upapplication.ymlfile.User fiels
nameandsurnamehas been renamed tofirstNameandlastName- it needs be updated if used in custom mail or document templates.Recommended version of MongoDB is updated to 4.2.3.
2.0.X to 2.1.X#
There is a significant change related to new Document Worker that handles generation of documents from templates and filled questionnaires. We need to run RabbitMQ and document-worker with correct configuration according to server, see Deployment with Docker and Configuration for details.
1.10.X to 2.0.X#
Changing the major version actually does not mean any problem in migration, it has been made due to significant internal changes (restructuring, new repositories, etc.)
If we are using Docker for running DSW, we need to change it according to new documentation of Deployment with Docker and Configuration.
Crontab image is no longer needed.
A DMP template configuration file must contain list of
allowedKMs(see the default root template).
1.9.X to 1.10.X#
Custom DMP templates needs to be upgraded to a new structure (see the default root template).
Compatibility#
Important
DS Wizard components (server, client, document worker, mailer, registry) should always use the matching version (compatibility is assured)!
The DS Wizard is compatible with all recent versions of web browsers Chrome, Opera, Firefox, and Edge. We do not recommend the use of Internet Explorer.
The following table shows the compatibility of the DS Wizard with the metamodel versions of Knowledge models, Document Templates, Project Importers, and the Registry.
DS Wizard |
KM Metamodel |
Document Template Metamodel |
Project Importer Metamodel |
Registry |
|---|---|---|---|---|
3.22.0 |
13 |
11 |
1 |
3.22.0 |
3.21.0 |
13 |
11 |
1 |
3.21.0 |
3.20.0 |
13 |
11 |
1 |
3.20.0 |
3.19.0 |
13 |
10 |
1 |
3.19.0 |
3.18.0 |
13 |
10 |
1 |
3.18.0 |
3.17.0 |
13 |
10 |
1 |
3.17.0 |
3.16.0 |
13 |
10 |
1 |
3.16.0 |
3.15.0 |
13 |
10 |
1 |
3.15.0 |
3.14.0 |
13 |
10 |
– |
3.14.0 |
3.13.0 |
13 |
10 |
– |
3.13.0 |
3.12.0 |
13 |
10 |
– |
3.12.0 |
3.11.0 |
12 |
9 |
– |
3.11.0 |
3.10.0 |
12 |
9 |
– |
3.10.0 |
3.9.0 |
11 |
8 |
– |
3.9.0 |
3.8.0 |
11 |
8 |
– |
3.8.0 |
3.7.0 |
10 |
7 |
– |
3.7.0 |
3.6.0 |
10 |
6 |
– |
3.6.0 |
3.5.0 |
9 |
5 |
– |
3.5.0 |
3.4.0 |
8 |
4 |
– |
3.4.0 |
3.3.0 |
8 |
4 |
– |
3.3.0 |
3.2.0 |
8 |
4 |
– |
3.2.0 |
3.1.0 |
7 |
3 |
– |
3.1.0 |
3.0.0 |
7 |
3 |
– |
3.0.0 |
2.14.0 |
7 |
3 |
– |
2.14.0 |
2.13.0 |
7 |
3 |
– |
2.13.0 |
2.12.0 |
6 |
3 |
– |
2.12.0 |
2.11.0 |
5 |
2 |
– |
2.11.0 |
2.10.0 |
5 |
2 |
– |
2.10.0 |
2.9.0 |
5 |
2 |
– |
2.9.0 |
2.8.0 |
5 |
2 |
– |
2.8.0 |
2.7.0 |
5 |
2 |
– |
2.7.0 |
2.6.0 |
5 |
2 |
– |
2.6.0 |
2.5.0 |
5 |
1 |
– |
2.5.0 |
2.4.0 |
5 |
– |
– |
2.4.0 |
2.3.0 |
5 |
– |
– |
2.3.0 |
2.2.0 |
5 |
– |
– |
2.2.0 |
2.1.0 |
5 |
– |
– |
2.1.0 |
2.0.0 |
5 |
– |
– |
2.0.0 |
1.10.0 |
4 |
– |
– |
1.2.0 |
1.9.0 |
3 |
– |
– |
1.1.0 |
1.8.0 |
3 |
– |
– |
1.0.0 |
1.7.0 |
2 |
– |
– |
– |
1.6.0 |
1 |
– |
– |
– |
1.5.0 |
– |
– |
– |
– |
1.4.0 |
– |
– |
– |
– |
1.3.0 |
– |
– |
– |
– |
1.2.0 |
– |
– |
– |
– |
1.1.0 |
– |
– |
– |
– |
1.0.0 |
– |
– |
– |
– |
FAQ and Deployment Notes#
Frequenty Asked Questions#
This section tries to cover the common deployment issues people have and suggest what next steps should be done.
Why something is not running; what should I do?#
Check what is not running using docker-compose ps and then use also docker-compose logs <service> to check what the issue is.
Why I cannot upload locales/templates and document generation fails?#
You probably have some issue with S3 configuration or its deployment. Also, check whether you have S3 bucket present with correct name.
Why I cannot download files from DSW or generate document preview?#
Your S3 is probably not accessible by users. The S3 URL configured in Server Configuration should be reachable so users can download something from the storage.
There is some issue with the PostgreSQL database; what should I do?#
Please use the PostgreSQL documentation to check the cause, various things may have happened… especially if you tried to upgrade the database version.
There is some issue with the MinIO S3 storage; what should I do?#
Please use the MinIO documentation to check the cause, various things may have happened… especially if you tried to upgrade the storage version.
I upgraded DSW and now it does not work properly, what should I do?#
You should always check Upgrade Guidelines before upgrading, be sure that you followed all steps. In case you forgot and it is not possible to fix it now, you will have to rollback from you backup and do it again by following the guidelines this time. In case you encounter an issue even though you followed the guidelines, that might a bug and please report it.
Document templates show “Unsupported Metamodel”, what should I do?#
You need to update your document templates so those are compatible with your DSW version, e.g. from DSW Registry. If those are your own document templates, you need to update them according to Upgrade Guidelines.
Deployment Notes#
You should be knowledgeable with at least basics of server management, service operations, work with Docker, and debugging issues (e.g. accessing Docker logs).
The deployment can vary significantly based on needs and available infrastructure, we cannot help with different kinds of deployments and technologies that we are not experts with.
The deployment example serves for local testing purposes and should not be used as is for production. An expert should deploy DSW for production while considering local needs and capabilities.
Never update production instance without backups and preferrably try the update procedure first on a testing environment.
Running DSW locally is not “free”, you need people, time, and infrastructure. With that in mind, consider what option is the most suitable for you.
Development#
DSW can be extended in many ways and new components and ways of integrations can be developed to support our needs. Besides the API available for everything that can be done in DSW, new integration questions and project importers can be implementet to get data from outside to DSW, or new document templates and submission services can be created to get the data outside of DSW in the desired form.
This section provides information on how to develop custom content for DSW to fully tailor the tool to our specific requirements.
Table of Contents
Metamodel Schemas#
As Data Stewardship Wizard evolves, the internal structures may change during the time. To support migration under the hood, we use metamodel versioning for KM and templates.
KM Package (.km file)#
File for import and export of Knowledge Models is a JSON file that contains all KM packages (lists of change events with additional metadata). The structure of events is versioned using the KM metamodel version number and migrations in DSW automatically update the KMs when needed. As said, files according to this schema can be exported from Knowledge Model List or Knowledge Model Detail and then used for Knowledge Model Import.
Metamodel Version |
Schema file |
Changes (brief) |
Since DSW |
|---|---|---|---|
13 |
New question value types |
3.12.0 |
|
12 |
Enhanced integration (e.g. widget type) |
3.10.0 |
|
11 |
Annotations and timestamps for events |
3.8.0 |
|
10 |
Integrations with item template |
3.6.0 |
|
9 |
Annotations |
3.5.0 |
|
8 |
Metrics and phases are part of KM |
3.2.0 |
|
7 |
KM name attribute removed |
2.13.0 |
|
6 |
Multi-choice question type added |
2.11.0 |
|
5 |
Move event |
2.0.0 |
|
4 |
Refactored KM, optional chapter text |
1.10.0 |
|
3 |
Changed integration question fields |
1.8.0 |
|
2 |
Changed phases representation |
1.7.0 |
|
1 |
Initial versioned metamodel |
1.6.0 |
Document Context#
Document Context is provided to the document templates by document worker. It contains all relevant data about project/questionnaire with replies, related knowledge model, author, and more. As KM evolves, the context may evolve as well. It is versioned using the Template metamodel version number. A document template must support the metamodel that is in the current DSW instance. It is needed to know how the document context looks like especially for Document Template Development.
Metamodel Version |
Schema file |
Changes (brief) |
Since DSW |
|---|---|---|---|
11 |
Change template metadata |
3.20.0 |
|
10 |
New question value types |
3.12.0 |
|
9 |
Enhanced integration (e.g. widget type) |
3.10.0 |
|
8 |
Annotations change |
3.8.0 |
|
7 |
Project tags and description |
3.7.0 |
|
6 |
Integrations with item template |
3.6.0 |
|
5 |
Annotations |
3.5.0 |
|
4 |
Metrics and phases |
3.2.0 |
|
3 |
Project versions |
2.12.0 |
|
2 |
Reply provenance |
2.6.0 |
|
1 |
Initial versioned metamodel |
2.5.0 |
Template (.json file)#
Each template has its descriptor file template.json which contains all the information about the template, its format(s) and steps how to produce the document(s). It is also versioned by the Template metamodel version number. This file also contains the actual number of the supported version… With local Document Template Development, we will need to manage the file according to the schema; however, when Document Template Editors are used, we will define it using forms directly in DSW.
Note
Between versions 1 and 5, the structure of template.json is still the same. Only the document context has been changed.
Metamodel Version |
Schema file |
Changes (brief) |
Since DSW |
|---|---|---|---|
11 |
Change template metadata |
3.20.0 |
|
10 |
New question value types |
3.12.0 |
|
9 |
Enhanced integration (e.g. widget type) |
3.10.0 |
|
8 |
Annotations change |
3.8.0 |
|
7 |
Project tags and description |
3.7.0 |
|
6 |
Integrations with item template |
3.6.0 |
|
5 |
Annotations |
3.5.0 |
|
4 |
Metrics and phases |
3.2.0 |
|
3 |
Project versions |
2.12.0 |
|
2 |
Reply provenance |
2.6.0 |
|
1 |
Initial versioned metamodel |
2.5.0 |
Document Template Development#
Document Templates allows to specify how to export a questionnaire in form of a textual file. It is a highly flexible element of the tool; however, the development requires basic programming skills with Jinja2 templating language. We can develop the document templates either on our local computer (traditional development with text editor or IDE) or directly in DSW using Document Template Editors.
Note
Requirements for Local Template Development
Your favorite text editor or IDE
DSW Template Development Kit (see below)
DSW instance (recommended to have local one) with your admin account
Python 3.7+ (with pip) or Docker
DSW Template Development Kit#
Our Template Development Kit (TDK) provides a simple way how to work with templates locally. It is a CLI tool written in Python.
Video Tutorial#
This is a comprehensive video tutorial on how to use the DSW Template Development Kit.
Installation#
You can install it easily using pip from Python Package Index (PyPI). Optionally, you can use virtual environment or other installation option described in the TDK repository.
pip install dsw-tdk
dsw-tdk --help
It is also possible to use datastewardshipwizard/dsw-tdk Docker image when you don’t have Python locally:
docker run datastewardshipwizard/dsw-tdk --help
Commands#
There are these basic commands:
new= create a new template project, it launches a simple interactive wizard for template metadatalist= list all templates (latest versions) from configured DSWget= download a template project with specified template ID from DSWput= upload the local template project to DSW (once or continually on-change when--watchflag is used)verify= check the metadata of the local template projectpackage= create a ZIP distribution package from the local template project (ZIP is importable to DSW via its web interface)
Default template directory is current one for put, verify, and package. But new and get will create a new folder according to the template ID if not explicitly set in other way.
You can use --help to find out details:
dsw-tdk new --help
Environment variables and .env file#
To avoid entering API URL and credentials every time, you can use environment variables:
DSW_API= URL of DSW API with which you want to communicate. Hover mouse over your profile name to find the About section where URL is specified.DSW_USERNAME= your login (email address) to that DSW instance, must be administrator accountDSW_PASSWORD= corresponding password
To make this even easier, you can store those in .env file in the project root and it will be loaded automatically. Or you can specify the path to a .env file:
dsw-tdk --dot-env /path/to/.env list
Template Metadata#
Each document template in DSW has metadata stored in template.json file:
id= composed full ID of the template (organizationId:templateId:version)organizationId= identifier of organization developing the template (lowercase, numerics, dot)templateId= identifier of template (lowercase, numerics, dash)version= version (semver) in X.Y.Z format where X, Y, and Z are non-negative numbersname= name of the templatedescription= short description of the templatelicense= name of the used licensereadme= longer description usually containing changelogmetamodelVersion= supported version of template metamodel, it affects with which DSW version is can be usedallowedPackages= list of package filters (see Package Filter) to specify supported packagesformats= list of available formats (see below Format) with specified steps for generation_tdk= TDK configuration for local development (not stored in DSW, see TDK Config)
DSW TDK handles id and readme for you, so you can skip then and naturally use README.md file separately.
Package Filter#
For filtering, the null value serves as wildcard, i.e., filter with all null values means that all packages are allowed.
orgId: identifier of organization (e.g.dsw)kmId: identifier of knowledge model (e.g.root)minVersion: minimal package version (in format X.Y.Z, inclusive)maxVersion: maximal package version (in format X.Y.Z, inclusive)
Format#
A template can describe how to produce several formats, each with these metadata:
uuid: UUID of the format (within template)name: display name of the formaticon: icon style (CSS classes), preferably Font Awesome, e.g.fas fa-file-wordsteps: list of steps for document worker to produce the document with this format, each step hasnameandoptions(see Steps)
TDK Config#
Those are local-only metadata used for development of the template. You can use them in versioned template.json but those are never stored directly in DSW.
version: metadata version for needs of migrationsreadmeFile: files used to get content forreadmeof the template, usuallyREADME.mdfiles: list of patterns to specify files that are part of the document template (it uses Git’s wildcard-match patterns, so you can also exclude files or directories)
Document Context#
Note
To work efficiently with the Document Context, you want to use object instead of the JSON-like one. Please read through DocumentContext.md directly (select different version if needed).
Document context is an object that carries all information related to a DSW questionnaire in order to produce a document. To investigate it, it is the best to use Questionnaire Report template with JSON format. The core fields are:
config= object with DSW configuration related to documents, e.g.,clientUrlfor referring to the DSW instancecreatedAt= timestamp when the document was createdcreatedBy= object describing author of the documentknowledgeModel= object describing used KM for the questionnairechapterUuids= list of UUIDs for chaptersintegrationUuids= list of UUIDs for integrationstagUuids= list of UUIDs for tagsentities= containsquestions,answers, and other maps with UUID-entity pairsname= name of the knowledge modeluuid= UUID of the knowledge model
level= current desirability level selected for the questionnairelevels= list of desirability levels possiblemetrics= list of available metricsorganization= object describing organization that runs the DSW instancepackage= object with metadata about the KM package such asversion,name, ordescriptionquestionnaireName= name of the questionnairequestionnaireReplies= map of replies with path-reply pairs, each reply hastypeandvaluequestionnaireUuid= UUID of the questionnairereport= object that contains report for the questionnaire that contains computed information about number of answered questions as well as metric valuesupdatedAt= timestamp when the document was last updateduuid= UUID of the document
Document Worker#
Document Worker component is used for document generation by supplying context to a specific template based on users demands. It retrieves a job to generate document, based on desired template and format it processed the input. This processing may be composed of several steps, usually some generation using Jinja2 and then optionally transformations. For processing Jinja2, we add several custom filters to those builtin directly in Jinja2.
Steps#
Each step of template produces output based on its (optional) input and options. For the first step, the input is the document context, for other steps, the output of the previous step is used.
json= produces a JSON as simply dump of document contextno options
jinja2= produces a document by supplying document context to specified Jinja2 template and renders it- options:
template= path of the template entry file (POSIX style, relative fromtemplate.json, e.g.template/index.html.j2)content-type= resulting content type of the rendered document (e.g.text/html)extension= file extensions for the rendered document (e.g.html)
wkhtmltopdf= runs wkhtmltopdf to transform HTML from the previous step to PDF documentno options
rdflib-convert= converts between RDF formats using rdflib- options:
from= source format (one of:rdf(XML),n3,nt,ttl,trig,jsonld)to= target format (as above)
Note
The steps including experimental are described in detail in steps directory (select different version if needed).
Jinja2 filters#
Note
All filters are described in JinjaFilters.md (select different version if needed).
To make template development easier, the document worker provides several additional filters:
any= check if any value of iterable is trueall= check if all values of iterable are truedatetime_format= formats datetime given in ISO format according to the given format stringextract= for object/map and list of keys it returns list of corresponding values from that object/mapof_alphabet= transforms a numer to letter of alphabet (e.g.0toa)roman= transforms given number to Roman numeralsmarkdown= transforms Markdown into HTMLdot= ends string with . if it does not already end with it nor is emptyreply_str_value= gets string value from given reply (if valid, otherwise empty string)reply_int_value= gets integer value from given reply (if valid, otherwise0)reply_float_value= gets float value from given reply (if valid, otherwise0)reply_items= gets list of items (their UUIDs) from given list-question reply (if valid, otherwise empty list)reply_path= joins given list of UUIDs into reply path
Jinja2 tests#
Note
All tests are described in JinjaTests.md (select different version if needed).
Tests can be used to make if conditions more readable using the is keyword. Just as in Python.
Graphics and Scripts#
If you want to include some graphics or JavaScript, we recommend you to put it directly into the HTML template file. In case of graphics, use base64 encoded content (suitable for smaller images like icons and logos):
<img src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot" />
Alternatively, you can of course reference picture that is accessible online. For JavaScript, again you can put there directly some script or reference it, for example, from some CDN:
<style type="text/javascript" src="https://code.jquery.com/jquery-3.3.1.min.js"></style>
<style type="text/javascript">
jQuery(".btn").click(function(){
jQuery(this).toggleClass(".clicked");
});
</style>
You can split your template code into multiple files and the use include directive that opens the file and inserts its content where the directive is placed - like we do for including CSS style in HTML template (only one complex HTML file is generated in the end):
<head>
<title>Data Management Plan</title>
<meta charset="utf-8">
<style>{% include "root.css" %}</style>
</head>
Local Template Development Procedure#
Prepare template project locally and run
dsw-tdk put -fwwith.envfile prepared for your dev instance.Open a project in the DSW dev instance and set default template and format to the one you are going to edit.
Edit the template as you need and save the changed files (TDK will update the template in dev instance).
Switch to browser, click “Preview” tab for refresh.
You will either compiled document or information about error that will help you to fix it.
It is recommended to save and check atomic changes in the templates as it makes it more convenient for eliminating bugs.
Template Metamodels#
Version 11 (since 3.20.0)#
Removed
recommendedPackageIdfrom template metadata andshortNametogether withcolorfrom steps.
Version 10 (since 3.12.0)#
New possible value types for value questions:
DateTimeQuestionValueType,TimeQuestionValueType,EmailQuestionValueType,UrlQuestionValueType, andColorQuestionValueType(no changes needed in existing KM-specific templates).
Version 9 (since 3.10.0)#
If you are using integration object, the
requestItemUrlis changed toitemUrl.Integrations now have type, where the new Widget Integration has a different fields than API Integration (see schema).
Version 8 (since 3.8.0)#
Annotations and integration HTTP headers are changed from dict-like object with string-string key and value to a list of string-string tuples. Be aware that now there can be more values with the same “key” but that is usually unlikely.
Version 7 (since 3.7.0)#
Added description and project tags to the questionnaire object (if you do not need them, nothing has to be changed in the template).
Version 6 (since 3.6.0)#
Integration item template replaced item name. In templates you probably need to rename for integrations the property
itemUrltoresponseItemUrl.
Version 5 (since 3.5.0)#
All KM entities has now annotations (key-value dictionary). If you do not want to use those in your template, no changes are required.
Version 4 (since 3.2.0)#
Levels are renamed into phases and are using UUIDs. Phases are as part of the KM in
knowledgeModel.entitiesof the context.Metrics are now also identified by UUID and part of the KM.
Version 3 (since 2.12.0)#
Additional metadata about each replies has been added and structure of reply is changed (extra
.valueneeded). In case you are using filters such asreply_str_valueno changes are needed.For integration reply, the type values are renamed
IntegrationValue->IntegrationTypeandPlainValue->PlainTypefor consistency.
Version 2 (since 2.6.0)#
Changed
questionnaireRepliesto use path-reply map and removed then redundantquestionnaireRepliesMapfrom document context.Replies for list question represented as list of UUIDs instead of size used for numeric indexing.
Version 1 (since 2.5.0)#
Initial version of metamodel, introduced in DSW 2.5.0 as start of versioning.
More Info#
Integration Questions#
DSW can be integrated with other services using so called integration question. The answer to that type of question does not contain only the answer itself but also a link to that external resource (which can be done, for example, using a persistent identifier). Therefore, these answers help clearly understand what the researchers use and promotes interoperability.
Examples of such integrations that are used within Common DSW Knowledge Model is FAIRsharing or ROR.
There are two ways of how we can connect DSW to these services:
API - using an API provided by the external service to search for the results
Widget - using a specialized widget implemented for the connection with the DSW
Table of Contents
Integration Question - API#
Integration question can be connected to an external resource using its API. We can then search for the results from the external service using the DSW questionnaire interface. When we select an answer it is not only the text (such as a name of the database), but also a link to the external service to the selected item. The whole flow is denoted in the following diagram.
How integration question connected to, for example, FAIRsharing API works.#
External Service Requirements#
If we want to connect an external service using the API there are certain requirements for it to make the connection to DSW possible.
Allows search using free text
There must be a way to send a search phrase to the API so that it can filter the results based on it
Returns a JSON response with a list of results
The response must be JSON so DSW can parse it
There needs to be a JSON list where all the items matching the search query are
It is possible to construct a link to the selected item
We also need to be able to construct the link to the item from the data we get in the response so we can provide it with the answer
Configuration#
The configuration is done in the knowledge model editor. First of all, we need to create a new integration and choose its Type to be API. Then, there are some metadata, such as ID, Name, or Logo URL.
Request Configuration#
In the Request section, we configure how to make an HTTP requrests to the external service’s API. For that, we need to configure the following (the specific values depends on how the API works):
Request URL - what is the URL where we want to send search requests
Request HTTP Method - what HTTP method should be used
Request HTTP Headers - some headers might be needed, such as
Accept: application/jsonto have a correct response typeRequest HTTP Body - if we need to send some HTTP body
Allow Empty Search - some APIs don’t work if we try to search with an empty string, turn this of it’s the case
There is a special property ${q} that we can use within those fields. The property represents the string that users type to the questionnaire. So for example, we can write Request URL as:
http://example.com/api/search?q=${q}
Response Configuration#
In the Response section, we configure how to process the JSON respnonse from the external service. For that, we need to configure the following:
Response List Field - where in the JSON response is the list of items corresponding to the search query
Respone Item ID - what field represents an item ID in the returned JSON
Response Item Template - how we want to present the result to the user
We can use Jinja2 templates (Ginger implementation) in Response Item ID and especially in Response Item Template to make the response item look better.
Secrets and Other Properties#
Sometimes, we might need to use some secrets (for example for authentication token), additional properties (such as API URL if we want to use different one for testing and production), or basically any information that we do not want to include in the knowledge model. In that case, we can define some properties in the instance settings.
We need to navigate to Administration → Settings → Knowledge Models and there is a field called Integration Config. It is a YAML organized by the Integration ID at the top level and key value pairs for each property.
We can fill some propertes in. So, for example, if the INtegration ID of our integration is ourIntegration we can write:
ourIntegration:
authorizationToken: "abcd"
apiUrl: "http://example.com/api"
Then, in the configuration of our integration, we can use these properties in the request configuration, so for example the Request URL can be:
${apiUrl}/search?q=${q}
And we can add a header such as:
Authorization: Bearer ${authorizationToken}
Note
These properties can be accessed only from the integration with matching ID.
Video Tutorial#
We have the following video tutorial showing how to set up the integration question using API.
External Resources#
Integration Question - Widget#
Integration question can be connected to an external resource using a widget integration. When there is this type of question, instead of writing an answer, reserachers click on Select button. It will open the widget where they can pick their answer and it is then sent back to the DSW. The whole flow is denoted in the following diagram.
How integration widget connected to, for example, DAISY works.#
Configuration#
The configuration is done in the knowledge model editor. First of all, we need to create a new integration and choose its Type to be Widget. Then, there are some metadata, such as ID, Name, or Logo URL, as well as the Widget URL which is the URL where the widget is deployed. The URL will be open in popup window when researchers click on the Select button when filling in the questionnaire.
Implementation#
The actual implementation is done using DSW Integration Widget SDK. We recommend reading the readme and explore the examples to understand how it works.
Project Importers Development#
Warning
Project importers are an experimental feature.
Project importers are can be used to import the data from an external resource to DSW questionnaire. The importer creates the replies based on the data, therefore it needs to know the structure of the knowledge model it is compatible with.
We can implement a project importer using DSW Importer SDK. It is a JavaScript library we can import and use its API for the communication with DSW. The installation and usage is described in the SDK readme.
Example Importers#
There are some importers already implemented. They are a good resource to see how to use the SDK:
Submission Service#
As administrators, we can configure submission services using Document Submission Settings. The configured HTTP request is then used when a user clicks Submit for an allowed document for submission and selected the desired submission service. The document is sent as a body of the request (or as multipart, based on the configuration) to the external service that should process it and return HTTP response with status code, and possibly also the Location header and some textual message.
Usually, we will need a simple proxy service to be developed that will accommodate this to API of some information system, database, storage, or other service. For example, such a proxy service will be able to receive the JSON documents from DSW, retrieve additional information through DSW API as needed, transform it to some other resulting artifact and store it in some local database that is used by other systems.
Example Submission Services#
There are some submission services already implemented and can be used to check the implementation possibilities:
Dummy Submission Service which just based on the headers returns example result or error.
Email Submission Service sends an email through SMTP connection with the submitted document attached (or processed in case of JSON).
Nanopub Submission Service allows to store a nanopublication (in RDF TRiG format) in the distributed network of nanopublication servers.
Contributing#
Interested in contributing to the DSW development?
Ideas#
If you have some idea (feature request) how to extend the DS Wizard, you can add it to our Ideas website or you can send us an email at support@ds-wizard.org.
Reporting#
Bugs#
In case you find some bug, please create an issue and provide requested information or contact us via email support@ds-wizard.org.
Vulnerabilities#
If you find an security issue within DSW, please create appropriate issue. However, never include sensitive information in the issue as it is publicly available. Such information (e.g. logs) send to us via e-mail support@ds-wizard.org.
Development#
Our projects are open source and you can contribute via GitHub (fork and pull request):
Note
Carefully read README and CONTRIBUTING files (if present) and also try to contact the main developer of the project for further details. You should follow the same code style, be DRY, and fit our overall architectures and structuring.
Test Policy#
Testing is essential to ensure the successful construction and implementation of DSW. It is necessary to keep tests updated together with new features and other changes in the code.
Each component may use its own test suite (unit tests, acceptance tests, integration tests), which shall be described in the CONTRIBUTING file within the corresponding repository. To test all components together, we have the E2E test suite (Cypress) that tests according to various use cases, i.e., what can a user do within DSW using its web user interface.
Whenever a new feature is developed, it must be covered by tests. For the E2E test suite, a specific sub-task is created in our JIRA when applicable. All components must pass all tests before releasing (including release candidate versions).
The release candidate versions are tested with OWASP ZAP. Eventual found vulnerabilities related to the code or dependencies (not deployment) are solved prior to the release or listed in Vulnerabilities if not possible to solve for the release.
In case that a bug is found, it must be analyzed why tests did not find it. If possible and needed, test suites are extended to cover these (and similar) bugs.
Vulnerabilities#
All known vulnerabilities are listed here so user’s can be aware of them and possibly avoid them.
Reporting#
Vulnerabilities should be reported using issues. To submit a private report, please send it to us via email: support@ds-wizard.org. Vulnerability issues are the top priority and resolved in the shortest time possible:
Accept vulnerability issue report (GitHub or email)
Verify and reproduce the issue, classify severity
Publish as known vulnerability
Design solution
Implement the change and release a hotfix
Move to solved vulnerabilities
Known Vulnerabilities#
No vulnerabilities has yet been found or reported.
Solved Vulnerabilities#
No vulnerabilities has yet been found or reported.
Basic Hints for Security#
Change or remove default users
Adjust default role after registrations based on your needs
Provide DSW through proxy with HTTPS (both client and server application)
Backup data regularly (e.g. daily)
Use secured SMTP (SSL)
Use strong passwords, esp. for administrator accounts
Miscellaneous#
Additional information related to DSW that might be useful.
Table of Contents
DSW Registry#
DSW Registry is a place where we publish knowledge models, document templates and locales. It is very easy to get those into a DSW instance and use.
DSW Registry with a list of knowledge models.#
We first need to connect our DSW instance to the DSW Registry in DSW Registry Settings. Once we have that, we can:
Markdown Cheatsheet#
Various text fields in DSW can be formatted by using Markdown formatting language. Here you can get a basic overview of what can be achieved with Markdown.
Basic Syntax#
These are the basic Markdown elements supported by all applications.
| Formatting | Markdown Syntax |
|---|---|
Heading 1 |
# Heading 1 |
Heading 2 |
# Heading 2 |
Heading 3 |
# Heading 3 |
Heading 4 |
# Heading 4 |
Heading 5 |
# Heading 5 |
Heading 6 |
# Heading 6 |
| Bold text | **Bold text** |
| Italic text | *Italic text* |
Blockquote |
> Blockquote |
|
1. First item 2. Second item 3. Third item |
|
- First item - Second item - Third item |
Code
|
`Code` |
|
|
--- |
| DS Wizard | [DS Wizard](https://ds-wizard.org) |
|
 |
Extended Syntax#
These elements extending the basic syntax are supported in DSW.
| Formatting | Markdown Syntax | ||||||
|---|---|---|---|---|---|---|---|
|
| Name | Value | | --- | --- | | Dataset 1 | 123 | | Dataset 2 | 211 | |
||||||
|
``` Codeblock ``` |
||||||
|
|
~~Strikethrough~~ | ||||||
|
- [x] Task 1 - [ ] Task 2 |
Help#
Still need help? Or you think that some topic is missing? Do not hesitate to contact us!
The best way to reach us is by sending an email to info@ds-wizard.org.
In case you found a bug, vulnerability, or have a generic question related directly to DSW tool – please create a GitHub issue and also visit Contributing section.
For feature requests, we recommend using the Ideas page.
Comments#
We sometimes want to leave comments to discuss things with our team or just reminders for ourselves. We can write comments to each question in the questionnaire.
Example of a comment.#
Add Comment#
To add a comment, we need to click on the comments icon by the question which opens a side panel with all the comments related to that question. Then, we can write the comment into the text box and submit.
View Comments#
When there are any comments for a question, the comments icon is changed. It has a yellow color and shows the number of unresolved comments for that question.
In the questionnaire menu bar, there is also a Comments tab, showing a badge with the number of comments everywhere in the questionnaire. If we open the tab, we can see a list of questions for each chapter where there are some comments. Clicking on the question there will bring us to that question and open the comments side panel.
Comment Threads#
Comments are organized into comment threads for better clarity. We can either start a new thread or reply in an existing thread if our comment is on the same topic.
When the thread is resolved, we can click on the ✔ icon in order to resolve it. Resolved threads can be later viewed by selecting View resolved comments. They can also be reopened if needed.
Editor Notes#
Besides comments there are also editor notes which work the same way as comments but they are visible only to project editors and owners. We can use editor notes to internal communication with our team while working on the DMP and then comments to gather the feedback from supervisor or reviewer.