Skip to main content

Technology

The Data Capability project does not focus on technology – either for database or management software. However, the toolkit still covers off broad categories that these components fall into

DataBase Management Systems (DBMS)

A DBMS is a system designed  for the definition, creation, querying, update, and administration of databases. Many different vendors offer solutions in this space including Oracle, Microsoft and IBM. In addition some software packages have their own proprietary DBMS technology embedded within the applications. This proliferation of DBMS technology is one of the drivers for both data integration and data warehousing.

Database management utilities

Database management utilities are tools or suites of tools to support the operation of the data life-cycle. Within this life-cycle will be collect, store, transform, use, archive and purge. Utilities may offer capabilities across all of these steps or within individual ones. Many of the tools are DBMS specific, however much tooling is also user developed and not of industrial strength.

Data modelling and model management

Data modelling technology comes in many forms. Sometimes it is a standalone tool, often it is part of a wider toolset that may include process, application and enterprise modelling. In other cases, it may be appended to data management utilities or even data quality tooling. It should provide the capability to create, modify, maintain, present and import/export models at all standard levels (conceptual, logical, physical) using standard notations (UML, ORB, XML, etc.) and integrate with process standards such as BPNM.

Business Intelligence (BI) software for reporting and analysis

BI is always supported by technology solutions both in terms of the Enterprise Data Warehouse and the analysis/visualisation components. These can be procured either separately or as a single packaged solution. BI solutions are becoming increasingly feature rich around analytics. Care should be taken to ensure these features genuinely support the business requirements and that capability in analysis and interpretation receives a similar investment.

ETL and CDC (Change Data Capture) tools for integration

These components have been grouped together although it represents a wide range of tooling from very simple to very sophisticated.  Some of these tools perform data transformation to reformat/repurpose data for other use/systems. Others provide data integration or ‘enterprise bus’ type capabilities.

Data quality and data cleansing tools

These tools mine, profile, analyse, clean, de-duplicate and report on (generally) structured data. This is a very crowded market space with a number of vendors offering extremely feature rich solutions.  It is one area of data management (along with DBMS) that absolutely needs aligned technology support. Attempting to manually operate this functionality on multiple, large and complex datasets with much interdependency is almost impossible. However, there is still much home-grown tooling (especially around data troubleshooting and basic data quality) as enterprise strength software solutions tend to be expensive and require support, training and commitment to make them successful.

Meta data management tools including meta data repositories

These tools can be thought of as data dictionaries. There are at least seven levels where meta data can operate and what is certain is that maintaining a library of meta data will clearly require technology support.  Again there are a wealth of tools available, some of which will be bundled with other technology components, others that are standalone. It is worth noting that much of this technology is now available as a cloud service. Organisations should consider the trade-offs between buying and building infrastructure and renting/leasing virtual services.