ADOxx Modelling Method Conceptualization Environment

,


Introduction
This paper is an extension of work originally published in 2016 IEEE 20th International Enterprise Distributed Object Computing Workshop (EDOCW) [1] The importance of Modelling Method Engineering is equally rising with the importance of Domain Specific Conceptual Modelling Methods and individual modelling approaches. In addition to existing (de-facto-) standards (e.g. Business Process Model and Notation (BPMN) [2], Decision Model and Notation (DMN) [3], Case Management Model and Notation (CMMN) [4]), a growing number of groups around the world design their individual modelling-methods (in accordance with the definition of such a method by [5], [6]for a variety of application domains.). This is often seen as necessary, when model-based approaches are transferred in new application domains and hence require adaptations for modelling methods. A simple sample can demonstrated using the well-known standard for business process BPMN. Although BPMN can be used to design a administrative process, such as sending an invoice, it cannot be used to design a simple production process like producing a chair. The successor relation that indicates that one activity follows the other does not have properties like distance to the station, which is not necessary when sending an invoice, but is of utmost importance, when producing a chair. When analysing more complex scenarios like a car manufacturer shop floor (we faced in projects DISRUPT [60], GO0DMAN [65]), the adaptation requirements for a modelling language like BPMN becomes quite complex. Hence, providing well-known model-based approaches requires the adaptation by e.g. introducing the concept "distance" between two activities.
Challenging question is, how to support the generation of modelling tools that can range from a minor adaptation like the one introduced above, till the complete realisation of totally new modelling approach like a cyber threat modelling for cloud computing .
The authors of [8] believe that supporting the automatic generation of modelling tools can open a new quality in ASTESJ ISSN: 2415-6698 information systems development for engineers and customized design as well as encourage the use of modelling languages that are fitting to the custom needs. Often customer needs are defined by desired features like visualisation, querying, simulation or configuration and transformation, which are applied onto the model. Individual solutions enable the generation of lightweighted solutions with targeted provision of features that reduce the developers' aversion against overloaded modelling languages and inflexible or expensive modelling tools.
The engineering of such modelling tools is a result of the so called "conceptualization process of modelling methods", which is disseminated by the world-wide community of OMiLAB [10]. The complexity lies in the mapping of language artefacts and their corresponding functionality to concrete implementable and deployable modelling tools. In addition, knowledge domains are divided into more refined and specialized subdomains, where each domain needs to be characterized by its own abstraction and refinement of concepts from the so-called "real world" objects from the "subject under study". Hence, in order to capture the relevant semantic primitives that address domain specific needs, it is necessary to involve both the method engineers as well as domain experts. Today, there are different approaches, guidelines and practices for the development of modelling tools available that do not consider the full lifecycle from the design and collaborative development of a modelling tool, which unavoidably leads to limitations and inconsistencies in the conceptualization [7]. We, hence, propose a guideline and corresponding tools supporting method engineers along the conceptualization process supporting all phases and ensuring collaboration among involved stakeholders.
Karagiannis proposes in [6] the Agile Modelling Method Engineering (AMME) framework. Authors of [9] propose the Modelling Method Conceptualization Process that based on AMME and guides the method engineers during conceptualization. The same authors in [35] propose a toolbox that supports this process. The work at the hand introduces an extended version of this toolbox so-called "Modelling Method Conceptualization Environment" as well as introduces corresponding services. Hence the paper introduces a productservice-system proposal for modelling method conceptualisation. Moreover the paper evaluates this product-service system in three European Research projects, and one additional in the context of an in-house research project, and discusses evaluation results.
In this respect, the remainder of the paper is structured as follows: Section 2 briefly presents results of our state of the art analysis from [9], revisits AMME, the Modelling Method Conceptualization Process. Section 3 outlines the toolbox Modelling Method Conceptualization Environment with an emphasis on new extension to ADOxx Library Development Environment, so-called ADOxx-JAVA-MM-DSL. Section 4 presents Modelling Method Conceptualization Services, Section 5 introduces evaluation cases and discusses the evaluation results, while Section 6 concludes the paper and gives an outlook on future work.

State of the Art in Modelling Method Definition and Development Approaches
We published the initial version of this state of the art analysis in [9]. In this paper, we revisit shortly it for completeness reasons.
We investigate, how the modelling language of modelling standards has been specified. We focus on four aspects specification of (1) abstract syntax-, (2) semantic-and (3) notation (concrete syntax) of the modelling languages as well as (4) samples provided to ease to understand modelling language specification and its usage.
The results of the analysis are presented in Table 1. In respect of abstract syntax specification organization used (1) graphical approaches and/or, (2) formal textual approaches and/or textual informal approaches. All of organizations utilize UML-Class Diagram [41] to specify abstract syntax of most of their modelling languages. In addition to UML-Class Diagram all standardization institute use natural language to specify the abstract syntax. Additionally, in order to ensure interoperability between systems using the given modelling standard and collaboration among parties utilizing the standard, most of the organizations attach importance to formal textual specification of abstract syntax either using with BNF [48] or XSD [49].
It seems that, regarding to specification of semantics there is no standard/approach used commonly by all organization. W3C uses formal languages such as RDF [50], Z Notation [54] [51] and standard mathematical notation. However, most of the organizations utilize semi-formal keywords defined in RFC-2119 [52] in order to define requirement level and constraints. Additions to these, all organizations use natural language to specify semantics of all modelling standards. Regarding to specification of notation (concrete syntax), obviously the modelling standards, which have been developed preferentially in the first place for human interpretation, precise has concrete syntax that is illustrated with images and described with natural language. On the other side, the standards, which have been developed for machine interpretation, have concrete syntax specified with formal textual approaches. Mostly, XSD is utilized for specification of concrete syntax of these standards. Finally, all the organizations choose to introduce features of their modelling standards with samples. The samples can be found in either in form of graphical or textual models.
According to analysis results, for the specification of selected modelling standards, the UML (UML Class Diagram) has been mostly utilized to specify the meta-model of the modelling language.   Figure 1 Approach, standards and artifacts used to specify modelling languages in control samples

UML as a Modelling Method Design Approach
As UML is utilized for specification of most of modelling standards in our set, we extend our literature research to analyze UML as a prominent modelling method, which is used during create and design phases of the conceptualization process. The pragmatic objective of this analysis is to understand if UML -in the context of modelling method design -covers all requirements for creating and designing a modelling method, or if there are any shortcomings that we have to consider during the development of our approach.
Glinz evaluates UML in [56] as a requirement specification language. He states that definition of requirements with UML Use Case Diagrams is possible but Use-Case-Diagrams alone are not sufficient. Definitions of concrete functional and non-functional requirements as well as definition of relation between the concepts in domain specific modelling method and requirements are not possible. The authors of [58] indicate the same problem and propose again a UML-based solution to define non-functional requirements and relation between the components in system by combining constructs from UML Class Diagram and UML Component Diagram. Hence the first shortcoming needs to be considered in our approach is; 1. Definition of functional and non-functional requirements and their relation between the concepts in modelling methods.
According to Selic [59] the domain specific modelling languages specify different viewpoints of a complex system. Hence, the complex system should be represented from different viewpoints and some features will be presented in several viewpoints. Moreover the language, presumably, will support multiple viewpoints for different sub-domains, which means language should allow use of different abstract and concrete syntax. With UML Class Diagrams the whole meta-model (abstract syntax) of language can be fragmented. But depicting all alternatives of abstract syntax together can make the class diagram very complex and hard to read. To the best of our knowledge the UML does not offer definitions of different concrete syntax for same concept of modelling language. Hence the next shortcomings are 2. Fragmenting whole meta-model into individual metamodels composing concepts for different sub-domains and still be able to define link between concepts in different individual meta-models 3. Having another abstraction layer to represent modules and layers of modelling language as well as relation among them without representing complexity of abstract and concrete syntax 4. Assigning different concrete syntax to the concepts in modelling language.
According to authors of [55], nowadays, models are used to elaborate design decisions, sort out different concepts and exchange the ideas in mainstream software development. They state the importance of traceability during the transformation of information, communication of decisions etc. from design into implementation and vice versa. They indicate that in order to establish the traceability the support of modelling environment is as essential as the approach itself. To the best of our knowledge UML itself does not support such traceability. Then the next shortcoming is: 5. Traceability of information (e.g design decisions, changes, suggestions etc.) during the knowledge exchange among experts within design phase and also from design to implementation vice versa.
Karagiannis and Kühn in[5] [57] argue that modelling methods have three major components (1) Modelling Language, (2) Modelling Procedure and (3) Mechanisms & Algorithms. Hence besides the modelling language of domain specific modelling method, we have to consider also designing modeling procedure and mechanisms & algorithms during the design of a modelling method. The design of modelling procedure and mechanisms & algorithms can be possible with using UML (e.g UML Activity diagram for design of the modelling procedure and if we consider description of mechanism and algorithms as sequence of object interactions and message exchange, the UML Sequence diagram can be used for description of mechanism and algorithms). Hence we would not see any shortcoming in UML for this issue. But in order to emphasize requirement of ability to design modelling procedure and mechanisms & algorithms besides the modelling language of a domain specific modelling method and requirement, and requirement of having corresponding supportive modelling environment, we would list this issue here rather as a general requirement than shortcoming of UML that we have to consider in our approach. Hence last but not least; 6. Possibility to design modelling procedure and mechanisms & algorithms of a domain specific modelling language.

Agile Modelling Method Engineering and Conceptualization Process
Having roots in software engineering, as it is in agile software development, during the modelling method engineering, involved stakeholders need procedures, structures and supportive tools allows high iterative process with as less as possible bureaucracy, and offers agile value and follows principles in Agile Manifesto [37] AMME is proposed in [6] to support modelling method engineering during propagation and evolution of modelling requirements. The OMiLab Lifecycle [10] instantiates AMME and defines the internal cycle of a modelling method conceptualization with five phases; (1) "Create" as a mix of goal definition, knowledge acquisition and requirements elicitation activities that capture and represent the modelling requirements; (2) "Design" specifies the meta-model, language grammar, notation and functionality as model processing mechanisms and algorithms; (3) "Formalize" aims to describe the outcome of the previous phase in non-ambiguous, formal representations with the purpose of sharing results within a scientific community; (4) "Develop" produces concrete modelling prototypes and finally (5) "Deploy/Validate" involves the stakeholders in hands-on experience and the evaluation process as input for upcoming iterations.

Figure 1 Modelling Method Conceptualization Process
Due to the involvement of several stakeholders with varying knowledge backgrounds, perspectives and different objectives, in the conceptualization of a modelling method, the authors of [9] propose so-called Modelling Method Conceptualization Process (as depicted inFigure 1) by adding additional feedback channels into the OMiLab Lifecycle between: (1) Create and Design, to prove, if the designed modelling language covers the identified application scenarios and considers the identified requirements; (2) Design and Formalize to ensure formal approval of modelling language, as well as (3) Design and Develop -to improve modelling language in earlier stages before it is released and deployed.

Modelling Method Conceptualization Environment
The work at hand introduces the "Modelling Method Conceptualization Environment from ADOxx.org. A toolbox (as its high-level architecture depicted in Figure 2) that initially has been introduced in [35] and that instantiates the above-mentioned conceptualization process and supports method engineers during each phase. The only exception is that of the "Create" phase, as this part is regarded as the most creative phase and standard tools and methods (also in some cases pen and paper can be the most appropriate tools) shall be freely selected. Modelling Method Conceptualization Environment proposes a combination of tools in sense of Integrated Development Environment (IDE), such as the Modelling Method Design Environment (MMDE, available to download and install at [11]) for the Design, the ADOxx Library Development Environment (ALDE) for Formalize and Develop, Adoxx.org Build, Test and Deployment Services (available at [22]) for Deploy/Validate Phases. As depicted in Figure 3, typical application scenario would be; during the create phase domain experts and method engineers come together, define goal of modelling method, acquire and elicit requirements, in design phase method engineers with tight collaboration of domain experts specifies the meta-model, language grammar, notation and processing functions on MMDE, as method engineers formalize design of modelling method collaboratively and commit on ALDE, developer(s) based on that formalization implements concrete modelling toolkit prototype within ALDE and ADOxx Development Toolkit. Developer(s) uploads the prototype into ADOxx.org build server, semiautomatic service behind starts with completeness check, building installation package, testing of installation package and optionally deploy it on selected developer's space to allow to download the toolkit, to be tested and validated by community members, so to get feedback from them or the build services simply sends a link to corresponding owner to download and/or share the modelling toolkit.
It is worth to mention that one of the objectives is to provide loosely coupled tools, so involved actors have the flexibility to decide to use one, a combination of tools from the toolbox or even use other appropriate tools of their choice, (e.g. method engineer uses MMDE during the Design Phase, but formalize the modelling In the following sub-sections current abilities of the tools from the environment are shortly presented.

Modelling Method Design Environment
The Modelling Method Design Environment (MMDE) is itself a modelling tool to design other modelling methods. MMDE has been implemented based on lessons learned from results of the state of the analysis, which is discussed in pervious chapter, and from the experience of the authors, who have been involved in more than 20 modelling method/tool development projects for varying domains. Based on these lessons learned, UML [12] has been identified as a starting point. Hence, the MMDE takes a subset of UML and extends it with required concepts and functionalities in order to overcome the following challenges (Ch), which are introduced after the state of the art analysis: Ch1-Definition of functional and non-functional requirements and their relation between the concepts in modelling methods; Ch2-Fragmenting the whole meta-model into individual meta-models composing concepts for different sub-domains and still be able to define links between concepts in different individual meta-models; Ch3-Having another abstraction layer to represent modules and layers of modelling language as well as relation among them without representing the complexity of abstract and concrete syntax; Ch4-Assigning different concrete syntax to the concepts in modelling language; Ch5-Possibility to design modelling procedure and mechanisms & algorithms of a domain specific modelling language.
To overcome Ch1, "Requirements" model type (as depicted in Figure 4) is implemented that allows the elicitation of requirements, specifying their status as well as dependencies among them. The described requirements in this model type can be referenced to (a) all the modelling classes modelled in the related model type "Meta-Model" classes, (b) graphical notation (concrete syntax) definitions modelled in the "Graphical Notation" model type, (c) the "Modelling Stack" definition and (d) to the functionalities modelled in "Mechanisms & Algorithms" models. For Ch2 and Ch3, we extend the class diagram from UML (as depicted in Figure 5) with concepts, so method engineers can differentiate between class and relation class as well as relate different meta-models (-fragments) with each other using "Weaving" techniques as they are introduced in [8] [9]. The modularization and layering of modelling language is essential to avoid complexities during the design of domain specific modelling methods [15][16] Hence, we propose representation of the Modelling Stack as the "Meta-models Stack model type (as depicted in Figure 6) allowing method engineers to differentiate meta-models in sense of different model types that target different fragments of the system.

Figure 6 A Sample Meta-model Stack Model
In order to target Ch4 and specify a proper graphical representation (concrete syntax) of each concept in a meta-model, we introduce another model type called "Graphical Notation" model type (as depicted in Figure 7) allows definition of concrete syntax of model types with specifying graphical representations for each constructs in metamodels. This model type allows the description of graphical representations with the assignment of concrete images in PNG, JPG or Bitmap format including a description of the functionalities in the notation (e.g. attribute-value dependent visualization, context related views)

Weaving object referencing to a concept defined in another meta-model
Weaving object referencing to a metamodel (in that case it is recursive)

Figure 7 Sample Graphical Notation Model
In order to target Ch5 to define the applicable modelling technique as steps and corresponding results we propose a model type called "Modelling Procedure" model type". The Modelling Procedure Model Type (as depicted in Figure 8) allows method engineers to define the steps with their required inputs and produced outputs, as well as the sequence of steps based on the inputoutput relationships, in order to introduce case specific proper usage of their modelling method.

Figure 8 A Sample Modelling Procedure Model
Based on this procedural view, concrete Mechanisms and Algorithms, can be derived and depicted as Sequence and Component Diagrams from UML (therefore these diagram types has been implemented as model types in MMDE). Further details about MMDE can be found in [9]

ADOxx Library Development Environment
The ADOxx Library Development Environment (ALDE) aims to enable parallel development of modelling tools libraries based on the designs deriving from Design Phase, merging different libraries and ensuring maintainability. As an experimental prototype ALDE is uses the Resource Description Framework (RDF) as a format for data interchange [17]  The Figure 9 depicts the architecture of ALDE. It is a development environment based on the Eclipse IDE [18] and includes a meta-meta-model defined in RDFS, the ALDE vocabulary.
Having the vocabulary and utilizing Apache Ant® [19] as a build mechanism, the environment enables the definition of the transformation processes from ADOxx Library Languages to RDF and vice versa. Moreover ALDE serializes libraries in an arbitrary RDF format; for the prototypical realization RDF Turtle [20] has been used (the Figure 10 depicts code snippets produced by the environment) and includes the RDF XTurtle Editor developed by [21]. Having libraries in RDF Turtle format and a RDF Turtle Editor available, method engineers can adapt declaratively and script libraries collaboratively using standard functionalities of source-code management systems. Merging several libraries or integration of parts of libraries in each other becomes possible. On the other hand, ALDE includes verification services to ensure that the newly developed, edited or merged libraries are consistent with ADOxx platform requirements.

Figure 10 Snipper of Class and Model Type Definitions in ALDE in RDF
The new extension to ALDE is a new DSL based on Java, which has a working title of "ADOxx-JAVA-MM-DSL". The ADOxx-JAVA-MM-DSL is developed according to feedback on

IDE
ToABL previous version of toolbox, which are presented and discussed in [35].
The ADOxx-JAVA-MM-DSL is a framework that creates several abstraction layers over the ADOxx Library Language (ALL) format, the ADOxx internal language that describes a metamodel [64]. Each layer simplifies and adds features to the bottom one. The framework gives, the possibility to operate and easily perform modification on a meta-model without dealing with its complexity. In order to assure that, an internal structure is managed that represents the ALL structure. This internal structure can be imported from an existing ALL meta-model.
All the constraints and rules present in the ALL syntax are managed, so the framework can guarantee that only syntactically valid ALL conform meta-models can be loaded and generated. The whole internal structure is an instance of java classes, so operations and definitions on the meta-model's concepts can be done using the java features. Parallel to that, additional utilities are integrated in order to bridge the gap between the development phases: • The compilation to the ALL meta-model to its binary format ABL: The ALL is a text based language used to describe a meta-model, in order to be imported and be usable in ADOxx it need to be compiled in its binary format ABL. A conversion engine is integrated into the framework. In such a way it is possible to automatize a previously manual step, required in order to create an automatized flow between formalization and development phase.
• The importing and deployment of the compiled metamodel in the form of ABL file, into a ADOxx Database: In order to create the respective modelling environment based on the created meta-model, is required to import the ABL compiled meta-model into the Database used by ADOxx. After this step the modelling environment relative to that meta-model can be executed and opened simply specifying at the launch time, the newly created database. The framework contains a feature that automatizes such a process, generating the Database and launching the modelling environment relative to the created meta-model. This step bridges the gap between formalization and development phase.
As depicted in Figure 11, the framework can be divided into three abstraction layers and two transversal layers of primitives and general features: The first abstraction layer is a package of java classes that exactly reflect the structure of the ALL syntax. This layer is responsible for converting each class in its ALL representation and applies syntax rules. Due to its closeness to the ALL syntax, working directly with this layer is difficult. The advanced of creating a meta-model using this layer instead using directly the ALL syntax is that all possible errors derived from miss-spelling and unhallowed sequences will be avoided, but the complexity in the definition of the meta-model remain the same as the ALL.
The second abstraction layer abstracts all the concepts of the First layer to a more design friendly way, providing the possibility to work directly with the concepts of Libraries, Classes, Relations and Attributes. These concepts are more familiar to method engineers that are familiar with the interactive development approach directly in the ADOxx Development Toolkit. This abstraction layer defines also some semantic rules over the syntactic rules provided by the first layer. The entire concepts created at this level will be mapped to the relative concepts at the first layer in order to reflect the ALL syntax. At this level, certain helpful features are also implemented, such as the possibility to merge two or more libraries or to import one or several classes from a library to another.
The third abstraction layer is a factory layer and contains predefined methods that generate empty libraries as a starting point for the definition of specific meta-models with classes and relations. The main entry point of the framework is also at this layer with the possibility to explore all the features of the development environment.
The transversal primitives layer gives a formal representation of all the primitives allowed in the ALL syntax. The primitives are used at every level in order to define data such as Identifiers or Attribute Values; The transversal features layer contains functions used at every level and the ALL parser. The parser is the component that allows an ALL to be read from a file and being instantiated and loaded into the framework in order to be extended or modified.
The ADOxx-JAVA-MM-DSL supports following three scenarios: Definition of a meta-model from scratch: Using the third abstraction layer, it is possible to generate an empty ADOxx library that is the best starting point to create own specific metamodel. Starting from that object the method engineer can add Classes and Relations and their respective Attributes to the metamodel by writing pure java code. Once the java code is executed, it generates the ALL file and/or compile it in the ABL file and/or directly create the ADOxx Database and execute the prototype of newly developed modelling toolkit based on defined meta-model.

Extension or modification of an existing meta-model:
Using the parsing module of the transversal layer, it is possible to load an ALL file into an instance of the java classes present in the second and First layer of the developer environment. After the ALL have been loaded is possible to extend, modify or manage it in the same way as creating from scratch. Certain methods to find specific Libraries, Classes, Relations and Attributes that the method engineer wants to work with are available. As in the previous scenario, after the java code is executed generates the ALL file and/or compile it in the ABL file and/or directly create the ADOxx Database and execute the prototype of newly developed modelling toolkit based on defined meta-model Merging of two or more meta-model in one: This scenario is a particular case of the first two. In particular, with using the framework, it is possible to merge two or more meta-model. As it is in a sample snippet depicted in Figure 12, thanks to the features present in the second layer is possible to import the whole concepts from a meta-model to

Third Layer Primitives Features
another or do a fine-grained import of specific concepts from each meta-model to another one. It is possible to import a class from a meta-model to another managing automatically all its dependencies like the presence of Super Classes. The merging scenario is supported in order to minimize the conflict that may rise in the merging of two incompatible meta-models, providing useful information to method engineer regarding how to correct occurred conflicts. Adoxx.org Build, Test and Deployment Services [38] are webbased services that allow method engineers of the ADOxx community to build verified, professional and installable distribution packages that can be distribute to interested stakeholders. The service combines and validates all available inputs, integrates all elements, compiles the necessary artefacts and signs the outcomes and creates the actual installer as a download archive.
As a collaboration space for the development and deployment phases, the concept of "Developer Spaces" has been introduced in ADOxx.org [23]. These spaces enable sharing of intermediate/release results, discussing development resources from all pre/past phases in the form of source code, snippet, examples and distribution packages with the community.

Modelling Method Conceptualization Services
In addition to the development tools described in the previous chapter, an appropriate service support is foreseen to support the modelling method engineers. The services are provided on the ADOxx.org portal, supporting a community of more than 1.300 modelling method engineers world-wide.

Download:
For the download, ADOxx.org provides the Meta Modelling Platform ADOxx in combination, Installation Instructions, Frequently Asked Questions, Startup-Package as well as a set of more than 30 available application libraries, which can be used to start with.

Get Started:
For getting started, ADOxx.org provides important readings, provides a Forum that is structured according active communities, lists tutorial and training events that are offered free of charge, provides tutorial material for both the studentsin form of guide samples and slidesas well as for the trainerin form of a trainer handbook and offers tutorial videos and webinars.

Development and Support:
For the development, ADOxx.org provides aforementioned tools and additional developer utilities, 3 rd parties add on like but not limited to simulation, documentation, dashboards or Web-APIs. A collection of 200 graphical representations that introduce the major elements conclude the development support.

Community:
For collaborative development within the ADOxx.org community a map is provided indicating the ten laboratoriesnine in Europe, one in Asia, indicating the hot spots of developers, the participating research institutes, a set of 24 modelling tools as a result of [66], and development spaces that enable a collaborative development and enable the use from solutions and tools from other projects. The operation of this service centre is provided via the portal, social media like Twitter, Facebook and LinkedIn, or via email. In justified cases an onsite support is possible, where either the method engineer is trained, supported or critical implementation steps are performed by the ADOxx.org service centre.

Evaluation
Given that usually each modelling method engineering case differs from each other in sense of complexity of domain, variety of aspects to be targeted, number of involved actors, to calculate quantified evaluation means is difficult, andto best of our knowledge -there is no similar conceptualization environment, hence, it is difficult to bench-mark our proposal and quantify the evaluation and provide statistically objective results. On the other hand, the most important tangible and objective evaluation result would be deployed and ready to use modelling toolkits, specification of modelling methods and communication of community members as proofs of concept. Those proofs of concepts for each are online and freely accessible (with exception of in-house project case). The links to access those proofs of concepts for each case are provided under regarding sub-section below.
The conceptualization environment introduced above has been applied in four different cases for evaluation: three EU-funded research projects in the domain of multi-stage manufacturing, eLearning and cloud computing and additionally in an in-house development project, in the area of decision modelling extensions. These cases have been selected since the involved partners have varying profiles and expertise in given domains, in development and in modelling method engineering. In the following sub-section we introduce the cases and their requirements in method engineering manner.

Case 1: Conceptualization of a Modelling Method for E-Learning:
The FP7 project Learn PAd [24] proposes a processdriven-knowledge management approach based on conceptual and semantic models for transformation of public administration organizations into learning organizations. Learn PAd proposes a model-driven collaborative learning environment. In this case, 4 domain experts and method engineers have been involved. In addition, two developer teams, each consisting of 4 developers worked on the implementation of the tool. The results of the conceptualization process of this modelling method can be found at Learn PAd Developer Space [25], as well as the developed prototypes [26] can be downloaded and feedback can be given.

Case 2: Conceptualization of Modelling Method for Cloud
Computing: The H2020 project CloudSocket [27] introduces the idea of Business Processes as a Service (BPaaS), where conceptual models and semantics are applied to align business processes with Cloud-deployed workflows [28]. In this case, 6 domain experts and method engineers have been involved, as well as two developer teams, one with 5 developers, the other one with 2 members. The results of the conceptualization process of this modelling method can be found at CloudSocket Developer Space [29], as well as developed prototypes [30] can be downloaded and feedback can be given.

Case 3: Conceptualization of Modelling Method for holistic Manufacturing System Management:
The H2020 project DISRUPT [60] deals with the integration of innovative technologies into a holistic manufacturing system and optimization of production flow. The DISRUPT projects needs a modelling method to describe manufacturing system from supply-chain level down to shop-floor level. In this case 2 domain experts, one requirement engineer and one method engineer have been involved. Preliminary results can be found on DISRUPT Developers Space [61].

Case 4: Integration of existing BPMN and DMN Modelling
Methods: The in-house project requires integration of an already implemented DMN Modelling Method into existing BPMN 2.0 realization as part of a commercial product. In this case, 3 domain experts and method engineers, and a team of two developers have been involved.
The evaluation process was enacted in the following steps: (1) Provisioning: the tools -of the toolbox have been provided to the stakeholders in the involved cases. (2) Team Formation: representatives, -of the stakeholders in the project created development teams consisting of domain experts and method engineers following the conceptualization process and developing tools individually. (3) Feedback Phase: individual results have been consolidated periodically through video conferences and workshops, constituting the evaluation results.

Feedback on MMDE.
Pro: It is possible to specify requirements and dependencies among them as well as tracing them; (2) to define modelling language fragments and modules, (3) layering the modelling language with navigational constructs; (4) definition of syntax, semantic and assignment of notation (concrete syntax); (5) definition of weaving among construct in different meta-models; (6) assignment of (multiple-) graphical notation (concrete syntax); (7) explicit definition of modelling procedure; Contra: It is not possible to define application scenarios and use cases, and design results can be exchanged solely using ADOxx specific formats or as static content (image, PDF or HTML). Hence, double effort in the design and in the formalisation and or development is currently necessary.
Outlook: The MMDE is currently updated, to offer an XML export, which then can be transformed into different formats like the one that is used for the ADOxx-Java-MM-DSL.
In addition, several improvements on the modelling language are implemented to (a) enable the design of user scenarios, (b) better describe the features of the modelling method and their corresponding components as well as (c) enable a more detailed representation of the method procedure enabling the mapping from components and the corresponding elements of the modelling method.

Feedback on ALDE.
Pro: it is possible to transform libraries in a machine as well as human interpretable format, ability to use reasoning algorithms, due to standard semantic formats; reduces complexity to edit, merge and maintain libraries.

Contra:
To take over results from Design Phase require manual steps.; it re-quires different transformation scripts for different meta-modelling technologies (such as ADOxx, EMF).

Outlook:
The semantic-based verification of meta model is seen as a useful extension of the ADOxx-Java-MM-DSL, hence an integration will be experimented. However, we see the necessary skill level for the meta model developer currently as inappropriate and tend not to follow this path.

Feedback on ADOxx-Java-MM-DSL:
Pro: It is possible to merge libraries and start libraries from scratch. Furthermore, the code base can be stored and versioned in a versioning system enabling several developers in parallel to work on one library. Built scripts enable the automatic generation and deployment of the tool.

Contra:
The current code maturity needs improvement and documentation, enabling also non specialists to handle the tool.
Outlook: This tool will be further improved and tested in two EU-H2020 research projects and will consequently be taught at the ADOxx.org Training Days and Webinars to achieve the required maturity.

Feedback on ADOxx.org Tool Packing Services and Developer Spaces.
Pro: It is possible to have an installation package to distribute to interested stake-holders, building your own community around the modelling method, and get feed-back from them.

Contra:
Setting up and handling issues of a certain Developer Space involves certain manual steps, such, as the interested stakeholder has to send an e-mail to the administrator with a request of an own Developer Space.
Outlook: This tool packaging service will be stepwise opened, so that the developer can also include own software components, which are then composed into a single tool package.

Conclusion and Outlook
In this paper we introduce a toolbox instantiating the Modelling Method Conceptualization Process, which supports agile modelling method engineering. The toolbox has been evaluated through an analysis of four different cases: three EU research projects and one in-house project. The evaluation results put forward that having an approach and a corresponding toolbox following the idea of model-driven engineering approach is effective in terms of transferring knowledge from the analysis of requirements up to the development of solutions. Being three main tools, MMDE, ALDE and ADOxx-Java-MM-DSL, prototypes that are at about Technology Readiness Level 5, hence lack of full integration or automatic data exchange ability, and the need of manual steps building Developer Spaces came out as major limitations of the toolbox. As an outlook the following items derived from the evaluation as future work: (1) currently we are evaluating development alternatives of DSLs with using Xtend [62] or RDF; building on existing work [63] in the field and integrating it into ADOxx-Java-MM-DSL, (2) enabling graphical modelling method design to transform into machine understandable format, (3) formalization of modelling method design using mathematical models such as FDMM [33] or Proof of Concept prototyping, (4) automatization of tooling services and deployment onto developer spaces, (5) full integration of tools within a holistic development environment.