Alderstone CMT is the fastest and most advanced data migration tool designed for the BMC Remedy platform.
Our unique approach to data migration includes a Remedy workflow expert system which analyzes your Remedy workflow tobuild a migration path tailored specifically for your BMC Remedy ITSM application and customizations, automated data mapping and a powerful migration and transformation engine capable of migrating data faster than any other Remedy data migration solution available today.
Let's review the technical challenges of Remedy data migration and our unique solution to these problems...
- Why is BMC Remedy data migration so difficult?
- The principals of data migration
- Full control over your data migrations
- Just how fast is CMT?
- CMT's flexble architecture
For additional technical information please register to download the Technical Overview fact sheet
Please sign up for one of our web demonstrations of CMT if you have any queries then please get in touch with us. Our team are very happy to talk over your business challenges and advise on how CMT can help.
Why is BMC Remedy data migration so difficult?
The BMC Remedy IT Service Management Suite has been the worldwide leader in IT Service management.
However, data migration has not traditionally been an area of strength for the ARS platform. Data migration is very important for companies performing upgrades, migrating to the cloud or managing ITSM as a service.
What are the key challenges when managing data migrations for BMC Remedy ITSM?
The BMC Remedy ITSM application is highly configurable and its behaviour is largely data-driven. The configuration data which drives the behaviour of the application is complex, and intertwined with other configuration data sets such as foundation data, including People, Support Groups, Sites and the Operational Catalog.
BMC Remedy ITSM has been developed over many years and has a variety of ways in which data is related. In some cases the Request ID is used, in others it is an Instance ID (GUID), or both, or neither. The variation in the relationship definition makes data model analysis complex and time-consuming.
Differences in software versions between environments may require transformation and/or mapping. Often this mapping is not one-to-one, for example, changes in the CMDB between ITSM 7.1 and ITSM 8.1 requires new Asset People records to be created for all People.
Remedy ITSM installations are frequently customized meaning there isn’t a one-size-fits-all standard for data migration.
Unlike some other enterprise-level software applications, Remedy does not enforce its data integrity using RDBMS concepts like foreign-keys or domains. Instead, Remedy enforces its relationships at runtime using Remedy workflow. There is no documented data model for the BMC Remedy ITSM application, rather the data model and relationships emerge organically from the workflow. Interpreting the data model from the Remedy workflow requires skilled technical analysis.
When migrating data from one system to another it is essential that referential integrity is preserved. This is made difficult by the extensive use of system-generated identifiers used in the application, specifically; Request IDs and Group IDs.
These unique identifiers are only unique on an individual Remedy instance and different Remedy instances will hold different data records that have the same unique identifier. Clashes in these unique identifiers, caused by careless data migration, often results in unexpected application behaviour and can break Remedy’s multi-tenancy security model.
Data Migration Principals
There are three key components that are common to all data migration solutions
What? Where? How?
What data to export, Where to put it, and How it should look when it gets there.
Mechanism for moving data
The process of defining the instructions to accurately migrate data is an expensive exercise that needs to be repeated for each new version of BMC Remedy ITSM. Furthermore, due to the ease with which Remedy applications can be customized, invariably each company using ITSM will have adapted both the source and target applications to meet their business requirements. A one-size-fits-all instruction set created for specific versions of BMC Remedy ITSM will not be effective in the real world due to these customizations.
CMT takes a unique approach to these challenges. The Alderstone CMT Expert System is capable of understanding a Remedy system and automating the creation of the Instructions. The CMT Expert system is a data driven rules-based engine that performs two key activities of Discovery and Analysis.
Discovery is the process by which CMT gathers the Remedy system meta-data needed to support data migrations.
CMT reads the Remedy data dictionary to obtain the meta-data for the Forms, Fields, Indexes, ITSM Modules and data relationships between Forms. This unique feature allows CMT to build an accurate data model specifically for your Remedy ITSM applications, even if you have heavily customized them. This collection of meta-data for a Remedy system is called a Version. The Version only contains ITSM meta-data and does not contain any application data.
Analysis is the process by which discovered meta-data is converted into the instructions that can be used by the Engine to perform data migrations.
CMT calls these instructions a Migration Package.
CMT takes a unique approach for identifying the data in your Remedy ITSM application which should be migrated; it builds a data model based on the relationships discovered by analyzing the Remedy application workflow. This migration data-model is called a Relationship Set, and describes all the Forms which need to be migrated as a tree, where relationships between Forms are described using Remedy qualifications. This dynamic approach for identifying and building the data set for migrations enables CMT to be used against even highly-customized Remedy applications.
There are a very high number of relationships between Forms in the BMC Remedy ITSM application. Unfortunately, the majority of these relationships are not useful for the purposes of data migration.
CMT uses an expert system and a system of extendible and data-driven weights and heuristics to convert the complex workflow in a Remedy ITSM application into a data model which can be used for data migration.
This analysis is entirely automated and significantly reduces the technical effort for data migration projects.
While CMT's primary focus is the migration of an entire customer data set, CMT is designed to be very flexible, enabling you to build any related set of data for migration between Remedy systems. For example, promote SRD configuration between Remedy systems with appropriate change control processes.
Using powerful templating and rules-based heuristics analysis, CMT automates the process of creating data mappings between all Forms and Fields in the migration data-model.
Automation of this process ensures high quality and significantly reduces the technical effort required to accurately migrate between Remedy systems.
For customized systems which may have extra fields or Forms, CMT will automatically identify and report on any data that cannot be mapped automatically. For example, where an enumerated value list has extra values or where there is no matching field in the target environment.
Data Mapping rules for all out-the-box ITSM versions are provided by Alderstone. CMT automates the process of applying these templates to your local environment.
For example, if you have customized your Remedy ITSM application to add fields to the Change Form and you have applied the same customization to your new ITSM 8.1 application, then CMT will automatically identify and map the fields together with no additional effort required. For more complex data mappings for customized applications, CMT’s data-driven rules are easily extended using the intuitive user-interfaces for mapping data.
CMT provides a full suite of transformation and data enrichment capabilities, allowing you to automate the remapping of Product or Operational Catalogs, or to apply business transformations, such as removing unused Sites.
The CMT data migration engine is responsible for executing the export, transformation and import of the data set defined in the Migration Pack.
CMT takes a unique approach for data migration; CMT uses native database tools to perform data migrations.
Even though data is migrated at the database level, the same rules which are enforced by the Remedy API are also enforced by CMT. For example, CMT will ensure that only valid enum values may be mapped as part of a data migration.
Database-level migration allows CMT to build highly performant multi-level joins to migrate complex and related Remedy data sets between environments. Migrating data at the database-level requires a profound understanding of the complex relationship between the Remedy application and the underlying database tables. CMT is designed to read and understand both the Remedy data dictionary and the ITSM application data model.
CMT leverages the power of the database for two key reasons;
The Remedy API is engineered primarily for transactional data manipulation and is not optimized for bulk data loading. Database-level import is significantly faster than data loading using the Remedy API. In real-world performance testing, a CMT database-level migration of 10,000 CTM People records took 5 seconds compared with over 20 minutes using the Remedy API. By using the database, CMT is able to achieve migration times which are orders of magnitude faster than Remedy API based migration tools.
The Remedy API permits dynamic joins to be created between Remedy Forms, however this feature is not performant for real world data sets.
The first step in migrating the complex data structures in the BMC Remedy ITSM application, is to identify all the data which should be migrated.
Taking a simple example; COM:Company is the form which holds the main record for a customer. We can trace a relationship between COM:Company and CTM:People via the Company field.
CTM:People has, in turn, a relationship to the CTM:People Permission Groups Form. To migrate a Company we need to identify the data across all these Forms.
A custom-written Remedy API utility can use API dynamic joins to identify the data across these three Forms. However, Remedy ITSM is far more complex than this simple example.
Frequently, more than ten Forms need to be dynamically joined to accurately identify the data to be migrated for an ITSM Company. The performance of multi-level Joins, using the Remedy API, against real-world data sets is very poor.
CMT discovers your Remedy systems, to create a Version snapshot of your Remedy application Forms, fields, indexes, and relationships.
CMT’s expert system creates the Migration Pack by analyzing the data model to create a Relationship Set. The expert system automatically maps the forms and fields in the source and target systems and defines data transformations to ensure a successful migration or upgrade.
Based on the instructions in the Migration Pack, the CMT Engine performs the migration of data applying transformations and enforcing data mapping and integrity rules. The data migrated by the Engine can be filtered at runtime, adding configurable prompts to the Migration Pack to collect inputs and filter data using Remedy qualifications.
Alderstone provide pre-built Migration Packs, data mappings and relationship rules for major out-of-the-box BMC Remedy ITSM releases. These can be used as templates for the upgrade of even heavily customized Remedy ITSM applications. Your Migration Packs are easily shared between CMT installations to ensure quality and consistency of data migrations across your organization.
Real-time reporting of progress and exception management are invaluable to operators managing data migrations. Built-in auditing of all tasks managed by the CMT Engine supports standards compliance.
Just how fast is CMT?
CMT is the world's fastest data migration tool designed for the BMC Remedy platform. Enabling you to migrate your Remedy data to a new system in a matter of hours rather than weeks or months using conventional tools.
The huge performance advantage of CMT not only reduces the duration and the costs of upgrades and data migrations but also makes you more agile on upgrade projects.
Being able to move data quickly gives technical staff the power and flexibility to respond quickly to unforeseen issues and to keep projects on track and on-budget.
Thoughput is the measure of how many records can be migrated within a specified time-period. The more records you can migrate the faster the migration and the less time you take to migrate a given number of records.
To measure throughput, we migrated 10,000 People records between two Remedy systems using three commonly-used legacy Remedy migration tools and using Alderstone CMT, and calculated the number of records per second (RPS)
Alderstone CMT accomplished a throughput of 4,000 records per second compared to the next fastest migration method (arx files) which achieved only 15 records per second.
|Third Party Tool||9|
Let's look at the performance of Alderstone CMT on a real upgrade project... One of our customers upgraded from a heavily-customized 7.6.04 ITSM to ITSM 8.1, and performed two data migrations;
- Migrate all data from the ITSM 7.6.04 system to ITSM 8.1 prior to UAT
- During the final outage cutover, migrate three weeks worth of data changes
How did CMT perform in this real-world upgrade?
- The full data set of 64 million records was migrated in only 10 hours.
- The delta data set of 6 million delta data records was migrated in only 2 hours.
But how does that compare with other data migration tools?
We calculated that BMC Migrator performing the same operations would require 87 days (running 24 x 7) to complete the full data migration and would have required an outage of 8 days to move the same delta data set!
The chart below compares the total upgrade migration time of CMT with the performance of BMC Migration for these operations - less is more!
|Migration Task||Alderstone CMT||BMC Migrator|
|Full Data Migration||10 hours||3 months|
|Cutover Data Migration||2 hours||8 days|
Upgrade data migrations are critical to your business. However the majority of data migration solutions for the BMC Remedy platform are so difficult for technical staff to use in the field, that they add unnecessary risks to business-critical operations.
For the majority of legacy BMC Remedy data migration tools, exception management is just logging to a file, and real-time reporting consists of checking the log files to see if the process has finished yet.
Technical resources find themselves trawling through huge log files to identify any potential data migration issues. During an upgrade cutover to Production, manual operations add unnecessary pressure and risk to the business-critical upgrade. Furthermore, very few data migration solutions provide any auditing capabilities. Using these solutions, it is difficult to tell what data was imported when and by whom.
CMT is different...it has been designed with the needs of technical staff in mind, providing the real capabilities that are needed to deliver quality data migrations.
Real time progress reporting
CMT’s Workbench enables you to manage and control migrations in real-time. With dynamically updated graphical tree views, full execution history and the migration overview consoles; CMT provides you with real-time visibility of the progress of data migrations. CMT allows technical staff to easily and accurately track, and communicate, the progress of business-critical data migration activities.
Exception handling & reporting
CMT’s sophisticated exception handling system ensures that all data migration issues are stored in the CMT Repository and reported clearly to the CMT Workbench; no more manually inspecting log files to see if something went wrong.
CMT’s Repository stores a full audit trail, tracking all data migration activities and updates to data in your BMC Remedy ITSM systems.
CMT has a three-tier architecture which can be installed in a variety of configurations to suit the security and infrastructure practices of your organization. For example, it can be installed onto a laptop or can be fully integrated into your data centre.
CMT has multiple security features designed to ensure that it can operate even in security-hardened environments. Including password encryption, a robust user-permission model and secure database access control, CMT ensures that the security model of your organization is enforced.
CMT does not require any changes to your existing Remedy applications. There are no changes to the workflow or data structure of your Remedy applications. The CMT user interface is completely standalone avoiding logistical complexity during upgrades and migrations.
The web user interface for the CMT application, hosted on the Tomcat Java Servlet Engine. The workbench does not perform any processor heavy activities which are all delegated to the CMT Runtime.
Persistent storage of all CMT data in a relational database.
Manages the execution of intensive processing activities including Discovery, Analysis and Data Migration. Polls the Repository for actions submitted by users via the CMT Workbench. The core engine uses Java wrapped as a Windows Service or UNIX Daemon for OS manageability.
Want to learn more?
Get in touch with us to find out more about how you can transform your management of BMC Remedy ITSM using Alderstone CMT.
Tell me more about CMT!
Request A Demo
Let us show you the power and effectiveness of Alderstone CMT at solving your ITSM data management challenges.
Download the CMT Data Sheets, release notes and much more.