Implementation Architecture
This final page consolidates what has been covered by the previous Wiki pages. The role of Magma Core and its strict use of a 'formally'* extensible model that can be consistently mapped to the 'real' world is to enable information systems to meet any information requirement (at least any that can be related to real activities, associations and material things that are 'real'). It is currently an embryonic project and, from initial use, has proved to be very powerful - both to ensure clarity of representation within and between systems and also to provide a lower burden of maintenance as requirements change and the need to consistently integrate data increases.
Magma Core and HQDM are only useful if information can be created, managed and used in a way that is useful to real users. This is where adopters of Magma Core will need to consider the intended use and adopt (and/or develop) a suitable application framework. As the data itself is in a highly structured form it can be used to serve many different applications, each with their own business and user contexts. The following diagram illustrates where Magma Core and HQDM fit into this type of application.
Conceptually there is no limit to the number of Magma Core instances and databases that are based on HQDM and serve the Magma Core instances. The model itself favours an immutable approach to data records (at least for application activities) and can support arbitrary separation of data and different rules around the sharing of reference and Information Operations data 'sets'.
The creation and use of Reference Data is of particular importance to the quality of the resulting Information Operations data and the applications that depend on the combined, integrated data perspective that Magma Core enables. The topic of Reference Data creation and management is significant but is not the focus of this wiki. The significant benefit that results from careful Reference Data management using models like HQDM is that it is far easier to share, extend in-use without (in the limit) any software updates and provides a major feature necessary for consistency in data representation. With Magma Core the reference data is 'just' data like all the other Information Operations data and can be queried over the same interfaces. Even where two systems are developed independently using Magma Core and HQDM there is a high chance of a low-cost mapping between those systems due to the framework provided by HQDM.
As noted elsewhere in the wiki there are some significant model patterns within HQDM that are worth adopting when familiarity with the model increases. Three particular ones are noted to flag their potential:
- Activity: Much of what we (humans) need information about relates to stuff that is involved with change. The activity pattern can unlock the ability to consistently represent change that is of interest in the past, present and future (such as plans that can be compared with records of what has 'actually' happened).
- Separation of names from what they represent: We are used to calling different things by the same name, and the same thing by different names... and being able to change the names we use. This pattern is a natural feature in HQDM. Although can be initially complex to adopt it is a repeatable pattern that can be used, in the limit, to remove entity names from all the other objects (not representing manes, or signs) in Magma Core. This can be powerful for the allocation (and change) of identifiers.
- Modelling systems in which parts can change 'independently' of each other.
Having mentioned the power of separating names from what they represent within the model, the use of the model does depend on some careful consideration of identifiers and parts of them (such as the component parts of URIs and IRIs). This is a topic that Magma Core and HQDM provides a basic treatment of but leaves the system considerations to adopters. A link to some background is available here: Linked Data IRIs. An issue has been raised to explore possible improvements to Magma Core and HQDM but the topic is flagged here to ensure that it is considered by adopters too.
The information management basis for Magma Core originates in the industrial applications for the lifecycle management of significant Assets (e.g. oil & gas industry facilities and related infrastructure). In particular, ISO 15926 was developed as a standards set for data integration (including data exchange) since the mid-1990s. Information management should take into account the lifecycle of the application domain; oil and gas infrastructure assets in the case of ISO 15926 (note that this is many decades, potentially even centuries). The reason for this extended view is that, similar to the need to be able to model real-world things to the widest possible extent, if the scope is restricted to just a part of the lifecycle it will inevitably place unfortunate constraints on the management of the data. This does not mean that all future information management requirements can be known, it just allows them to be accommodated when they are identified.
By considering lifecycle information management to this degree it does suggest that some wider changes can follow:
-
Not deleting or changing records in-use.
One challenge with managing corporate records is ensuring that the availability of data and any changes are managed well. However, a formally integrated approach can result in records being treated as immutable (at least for most users). This can offer benefits for accreditation, audit and compliance purposes.
-
Allowing a machine-readable rendering of management rules.
This is an idea that would transform information lifecycle management but is not formally achievable without a grounded approach to what is being represented by the data. Doing this would require a codifiable framework (implementable in logic with machine interpretable rules) but the use of a Top Level Data Model provides a foundation for such approaches.
-
Adoption of integration data modelling by most (if not all) connected systems in an low-loss way.
Reducing ambiguity in data through formal consistency methods is desirable for organisations that depend on information that they create. This could be a transition that happens over ~10–20 years, in a managed way, but without it the ability to formally manage corporate records that are extensible won’t be achieved.
Ultimately the use of Magma Core and HQDM is to enable consistent representation of data within and between systems in ways that haven't been adopted widely yet. As data quality is being increasingly recognised as the foundation for information that is fit for the purpose of supporting decision, this project hopefully offers the opportunity for anyone to explore and adopt models like HQDM that address the consistency challenge.
* This is not quite as formal as it would need to be to be in a "Formal Systems" sense but it is far closer to that than other systems that are in common use today. For information on other work relating to formal aspects and an initiative that is constructing a model that is closer to that goal look at the National Digital Twin programme's development of an Information Management Framework that uses a Foundation Data Model that is similar to HQDM.