Skip to main content

Validation errors in data migration

This article explains the validation errors in Data Migration.

Introduction

Data access in Fen-X is manage through Access Layers feature. This feature allows to restrict which user has access to which piece of data by creating "layers". Layers are records that hold the definition and relation between a key and the kind of data that it can be used on.

Under the hood the process consists in 3 main parts:

  • Define a key

  • Setup a piece of data to require that key

  • Grant that key to users

Access Layers To Data Access Definitions Translation

DataMigration uses Data Access Definitions to define data access. This definitions are generated based on existing access layers at the moment of the migration creation.

A quick table with terms mapping would be:

Access LayersData Access DefinitionsData KeyAccess KeyTypeAccess TypeData TypeData Level (see bellow for more info)

  • Access Key: any sequence of characters that will be required by a piece of data in other to be able to access it.

  • Access Type: indicates what type of access the key grants. Possible values:

    • Geographic

    • BusinessRelated

  • Data Level: restricts where the access key can be required based on the data level of the data hierarchy. It is composed of a subset of the "Data Type" (allowed targets) options in access layers that are translated into DataMigration equivalents:

    • Entity: becomes RootEntity data level

Example:

If we have a data access definition with the next information:

Access KeyAccess TypeData LevelsomeKeyBusinessRelatedRootEntity

This means we can configure a RootEntity (e.g.: Individual, Company) to require "someKey" with type "BusinessRelated" in order to be accessed.

EntityAccess Table

Entity Access is a constant but optional table included in each new migration to allow to setup required access to entities.

An example of the csv content for this table would be:

"parentAlternateId","alternateId","parentTableName","accessKey","accessType" "B378219B-CC4F-4BCD-8806-4C8CA4E51113","EE3FD77F-DD69-4D1D-912E-CF0905AA3A33","Individual","Global","Geographic" "3961042F-248D-4763-8BD8-4187D5AF4FB1","0E1A87D4-2B0A-4A14-B747-4F79860CC0B5","Individual","Enterprise","BusinessRelated" "F0621160-F54A-4EC7-848A-3A19F149E269","60721E8B-691F-473A-8E30-8FFBA777B701","Company","Enterprise","BusinessRelated" "FC4AC252-7B7F-405E-860C-DF5A68ED002B","60692BD2-4DC1-416A-A145-CB593B4D8ECC","Individual","AnotherKey","BusinessRelated"

As brief description of the columns in this table:

  • parentAlternateId: alternateId of the taget entity in the parent table we want to setup required access.

  • alternateId: unique sequence of characters that identify the row during migration process. (only used for error messaging at the moment of writting)

  • parentTableName: name of the table the target entity belongs to.

  • accessKey: value from Access Key column in Data Access Definitions.

  • accessType: value from Access Type column in Data Access Definitions.

Validation

During validation, the Data Access Definitions will be used to ensure the Entity Access table records hold allowed values.

Possible validation errors:

  • "...Value not found....": when any of the columns have no value. All the columns are mandatory for Entity Access table.

  • "...Table not found...": when the value in parentTableName does not exist or is not used in the current migration.

  • "...Data access not found...": when the combination of key type and datalevel does not exists in Data Access Definitions.

  • "...Required data level not found...": when the access key and acces type exists but they have not been configured to allow right data level.

  • "...Duplicate row found...": when either [alternateId] or [accessKey,accessType,parentTableName,parentAlternateId] combinations are duplicated in different rows.

Example error message:
”Validation errors found in csv files.,,10/18/2024 16:53:30,10/18/2024 16:51:47,5ee499de-805a-446f-8bd8-01fb1ad917ba,c8b1e193-2ff3-4da4-bf4f-b68b0ce0dddd
Validation,"EntityAccess: parentTableName,accessKey,accessType: Data access not found - ….. not found in Data Access Definitions when validating access for entity on table Individual. Check Data Access for missing access definitions or ensure the access provided is correct.",Ind-Bus-Confidential-Ent,10/18/2024 16:53:30,10/18/2024 16:51:47,5ee499de-805a-446f-8bd8-01fb1ad917ba,cd8824ea-ed88-4c08-ac57-5bdb32c141ee
​
The migration schema should be checked to troubleshoot such a validation issue. This artifact can be sourced using the below migration API.
​API Docs: GET Artifact Content

The fenx-data-migration-schema.xlsx is a useful document. The document includes a tab called Data Access Definitions. This should be checked for irregularities in relation to a validation error such as the above example.


​

Did this answer your question?