Skip to main content

Data Lineage Connection

Use Alteryx One to transfer lineage metadata from Server to an external data catalog. Each workspace supports a one-to-one connection between a Server and a catalog.

Caution

To enable data lineage functionality, the Service Principal used for communication between Server and Alteryx One must be assigned the Admin role. Custom roles based on the Admin role with reduced capabilities are not currently supported.

As a result, the Service Principal temporarily requires broader permissions than intended. This limitation affects only Alteryx One and does not impact the Server or any code running on it. We are aware of this issue and plan to resolve it in a future update.

Before You Begin

Ensure that the Alteryx One IP addresses are whitelisted. These IP addresses can be found on the Get Started with Alteryx One Platform documentation page.

We provide data lineage information for Alteryx Server runs in the OpenLineage format, an open standard for capturing pipeline metadata. This allows seamless integration with key consumers of OpenLineage events, including Collibra and Atlan, supporting governance, discovery, and collaboration. Note that Collibra and Atlan are not configured or managed by Alteryx.

For more information on supported tools and connection types, go to Supported Tools for Column-Level Lineage.

Data Lineage Security

No lineage information is stored either on Server or on the Cloud. Data only flows in transit, and this is encrypted using TLS encryption.

Data Sent from Alteryx Server to the Lineage Cloud Service

When a workflow is executed on Alteryx Server, we collect data lineage information at the workflow level.

When a workflow is executed, the following events take place:

  • START: The workflow execution begins.

  • RUNNING: The workflow enters a running state. It might remain in this state or loop until it moves forward to a result.

  • Outcome States: After running, the workflow can transition into one of several possible outcomes:

    • COMPLETE: The workflow finishes successfully.

    • ABORT: The workflow is intentionally stopped before completion.

    • FAIL: The workflow encounters an error and fails.

For each event, we collect the relevant metadata needed in order for the Lineage Cloud Service to create the OpenLineage events.

Examples

START Event Sent on Start of Execution

{
    "eventTime": "2024-11-22T00:00:00Z",
    "eventType": "START",
    "job": {
        "name": "TestWorkflow"
    },
    "run": {
        "runId": "00000000-0000-0000-0000-000000000000"
    }
}

RUNNING Event Sent on Establishing Input/Output Datasets

{
    "eventTime": "2024-11-22T00:01:00Z",
    "eventType": "RUNNING",
    "run": {
        "runId": "00000000-0000-0000-0000-000000000000"
    },
    "job": {
        "name": "TestWorkflow"
    },
    "inputs": [
        {
            "name": "input_a.csv",
            "namespace": "file"
        }
    ]
}

COMPLETE Event Sent on Successful Execution

{
    "eventTime": "2024-11-22T00:04:00Z",
    "eventType": "COMPLETE",
    "run": {
        "runId": "00000000-0000-0000-0000-000000000000"
    },
    "job": {
        "name": "TestWorkflow"
    }
}

FAIL Event Sent on Failed Execution

{
    "eventTime": "2024-10-21T14:30:00Z",
    "eventType": "FAIL",
    "run": {
        "runId": "00000000-0000-0000-0000-000000000000"
    },
    "job": {
        "name": "TestWorkflow"
    }
}

Additional Metadata

Once these events are triggered, AlteryxService consolidates all the relevant events and provide the additional metadata for final send off to the Lineage Cloud Service, such as:

  • job.name: A concatenated string containing the workflow name and the service app ID.

    Example: GenerateSalesDetail|01JTKMDMZZPXDD8NY51XKR5A5F

  • Owner: The user ID of the workflow owner.

  • Studio: The subscription ID associated with the workflow’s studio (only for manual runs).

  • Server URL: A Server URL that sends you to the workflow detail page in Server UI.

    The serviceId corresponds to the appInfos.serviceId field, which maps to the AS_Applications.id field (or AS__application_id in the AS_Queue record).

  • ServerJob.id: The unique job identifier.

Data Sent from the Lineage Cloud Services to the Lineage Consumer

Once the Lineage Cloud Service has received the metadata from Alteryx Server, it sends the relevant formatted OpenLineage events to the Lineage Consumer.

Examples of Dataset-Level Lineage Information

Open Lineage START JSON Example

{
  "eventType": "START",
  "eventTime": "2024-12-20T13:39:37.554Z", 
  "run": {
    "runId": "acde070d-8c4c-4f0d-9d8a-162843c10333" 
  },
  "job": {
    "namespace": "Alteryx-demo",
    "name": "Lineage Connection Workflow|5ea20186a171a51bc4c37155", 
  },
  "producer": "us1.alteryxcloud.com/lineageservice:0.0.1 ",
  "schemaUrl": "https://tools.ietf.org/html/rfc6901"
}

Open Lineage COMPLETE JSON Example

{
  "eventType": "COMPLETE",
  "eventTime": "2024-12-20T14:00:37.554Z", 
  "run": {
    "runId": "acde070d-8c4c-4f0d-9d8a-162843c10333" 
  },
  "job": {
    "namespace": "Alteryx-demo", 
    "name": "Lineage Connection Workflow|5ea20186a171a51bc4c37155"
  },
   "inputs": [{
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.AYX_COLORADO_STORES_SUMMARIZE"
  },
            {
    "namespace": "file",
    "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.yxdb"
            }],
  "outputs": [
    {
        "namespace": "file",
        "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.yxdb"
      },
    {
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.LAINE_TABLE"
  }],
  "producer": "us1.alteryxcloud.com/lineageservice:0.0.1 ",
  "schemaUrl": "https://tools.ietf.org/html/rfc6901"
}

Open Lineage FAIL JSON Example

{
  "eventType": "FAIL",
  "eventTime": "2024-12-20T14:00:37.554Z", 
  "run": {
    "runId": "acde070d-8c4c-4f0d-9d8a-162843c10333" 
  },
  "job": {
    "namespace": "Alteryx-demo",
    "name": "Lineage Connection Workflow|5ea20186a171a51bc4c37155"
  },
   "inputs": [{
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.AYX_COLORADO_STORES_SUMMARIZE"
  },
            {
    "namespace": "file",
    "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.yxdb"
            }],
  "outputs": [ 
    {
        "namespace": "file",
        "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.yxdb"
      },
    {
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.LAINE_TABLE"
  }],
  "producer": "us1.alteryxcloud.com/lineageservice:0.0.1 ",
  "schemaUrl": "https://tools.ietf.org/html/rfc6901"
}

Open Lineage ABORT JSON Example

{
  "eventType": "ABORT",
  "eventTime": "2024-12-20T14:00:37.554Z",
  "run": {
    "runId": "acde070d-8c4c-4f0d-9d8a-162843c10333" 
  },
  "job": {
    "namespace": "Alteryx-demo",
    "name": "Lineage Connection Workflow|5ea20186a171a51bc4c37155"
  },
   "inputs": [{
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.AYX_COLORADO_STORES_SUMMARIZE"
  },
            {
    "namespace": "file",
    "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.csv"
            }],
  "outputs": [
    {
        "namespace": "file",
        "name": "E:/ProgramData/Alteryx/Service/Staging/State Population Output.csv"
      },
    {
    "namespace": "snowflake://tjb31106",
    "name": "AYX_LINEAGE_DB.ELLENS_DATA.LAINE_TABLE"
  }],
  "producer": "us1.alteryxcloud.com/lineageservice:0.0.1 ",
  "schemaUrl": "https://tools.ietf.org/html/rfc6901" 
}

Examples of Column-Level Lineage Information

Open Lineage COMPLETE JSON Example

{
    "eventType": "COMPLETE",
    "eventTime": "2025-10-06T17:46:27.695Z",
    "run": {
        "runId": "173fc3f9-1501-4749-beae-32b56796596e",
        "facets": {
            "nominalTime": null,
            "parent": null
        }
    },
    "job": {
        "namespace": "AlteryxTest",
        "name": "ProductSales|68c48138430ae827850e8498",
        "facets": {
            "documentation": null,
            "sourceCodeLocation": {
                "_producer": "https://ayx-sandbox.bender.rocks/data-lineage-service/v1/datalineage",
                "_schemaURL": "https://openlineage.io/spec/facets/1-0-1/SourceCodeLocationJobFacet.json",
                "type": "Alteryx Server",
                "url": "http://AYX-LT-5MGCB54/gallery/#/app/app/68c48138430ae827850e8498/?type=service"
            },
            "sql": null,
            "jobType": null,
            "ownership": {
                "owners": [
                    {
                        "name": "68b9acee6065b0e905f38ea3",
                        "type": "USER"
                    },
                    {
                        "name": "68b9acee6065b0e905f38ea0",
                        "type": "SUBSCRIPTION"
                    }
                ],
                "_producer": "https://ayx-sandbox.bender.rocks/data-lineage-service/v1/datalineage",
                "_schemaURL": "https://openlineage.io/spec/facets/1-0-1/OwnershipJobFacet.json"
            },
            "serverJob": {
                "id": "68e40070b8565a17ce0ac2e7",
                "_producer": "https://ayx-sandbox.bender.rocks/data-lineage-service/v1/datalineage",
                "_schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json#/definitions/BaseFacet"
            }
        }
    },
    "inputs": [
        {
            "namespace": "file",
            "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\Products.xlsx",
            "facets": {
                "documentation": null,
                "schema": null,
                "dataSource": null,
                "description": null,
                "lifecycleStateChange": null,
                "columnLineage": null,
                "symlinks": null
            },
            "inputFacets": {},
            "outputFacets": null
        },
        {
            "namespace": "file",
            "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\OrderDetails.xlsx",
            "facets": {
                "documentation": null,
                "schema": null,
                "dataSource": null,
                "description": null,
                "lifecycleStateChange": null,
                "columnLineage": null,
                "symlinks": null
            },
            "inputFacets": {},
            "outputFacets": null
        }
    ],
    "outputs": [
        {
            "namespace": "file",
            "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\TotalSales.csv",
            "facets": {
                "documentation": null,
                "schema": null,
                "dataSource": null,
                "description": null,
                "lifecycleStateChange": null,
                "columnLineage": {
                    "_producer": "https://ayx-sandbox.bender.rocks/data-lineage-service/v1/datalineage",
                    "_schemaURL": "https://openlineage.io/spec/facets/1-2-0/ColumnLineageDatasetFacet.json",
                    "fields": {
                        "OrderID": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\OrderDetails.xlsx",
                                    "field": "OrderID",
                                    "transformations": [
                                        {
                                            "subtype": "IDENTITY",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "Category": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\Products.xlsx",
                                    "field": "Category",
                                    "transformations": [
                                        {
                                            "subtype": "IDENTITY",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "NewCount": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\OrderDetails.xlsx",
                                    "field": "Quantity",
                                    "transformations": [
                                        {
                                            "subtype": "TRANSFORMATION",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "ProductID": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\Products.xlsx",
                                    "field": "ProductID",
                                    "transformations": [
                                        {
                                            "subtype": "IDENTITY",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "TotalSale": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\OrderDetails.xlsx",
                                    "field": "Quantity",
                                    "transformations": [
                                        {
                                            "subtype": "TRANSFORMATION",
                                            "type": "DIRECT"
                                        }
                                    ]
                                },
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\Products.xlsx",
                                    "field": "UnitPrice",
                                    "transformations": [
                                        {
                                            "subtype": "TRANSFORMATION",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "ProductName": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\Products.xlsx",
                                    "field": "ProductName",
                                    "transformations": [
                                        {
                                            "subtype": "IDENTITY",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        },
                        "OrderDetailID": {
                            "inputFields": [
                                {
                                    "namespace": "file",
                                    "name": "C:\\ProgramData\\Alteryx\\Service\\Staging\\45816_d3034dfd34e946a48485299407c307a0\\_externals\\1\\OrderDetails.xlsx",
                                    "field": "OrderDetailID",
                                    "transformations": [
                                        {
                                            "subtype": "IDENTITY",
                                            "type": "DIRECT"
                                        }
                                    ]
                                }
                            ],
                            "transformationDescription": null,
                            "transformationType": null
                        }
                    }
                },
                "symlinks": null
            },
            "inputFacets": null,
            "outputFacets": {}
        }
    ],
    "producer": "https://ayx-sandbox.bender.rocks/data-lineage-service/v1/datalineage",
    "schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent"
}

Data Stored in the Lineage Cloud Service

We only store the configuration information that is used to establish the connection between the Lineage Cloud Service and the Lineage Consumer.

Data retention period: Indefinite.

Example

Atlan Configuration

  • Open Lineage Endpoint: https://alteryx.atlan.com/v1/lineage

  • Open Lineage Namespace: Alteryx-Atlan

  • Auth Method: Bearer Token

  • Bearer Token: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiMTIzNDU2IiwidXNlcm5hbWUiOiJ0ZXN0X3VzZXIiLCJleHAiOjE3MTQ5MDYwMDB9.SflKxwRJS

Collibra Configuration

  • Endpoint: http://localhost:8888/debug.test

  • Namespace: Alteryx_Collibra

  • Auth Method: None

Set Up Data Lineage Connection

To set up your data connection lineage, go to Admin Console > Data Lineage Connection.

Data Consumer: External Data Catalog

Set up, edit, or delete a data catalog connection.

Set Up Data Catalog Connection

To enable lineage metadata transfer from Server, set up a data catalog, such as Atlan or Collibra.

  1. In the menu, go to Admin Console > Data Lineage Connection.

  2. Under Data Consumer, select Set Up for an External Data Catalog.

  3. A Data Catalog Setup page opens.

  4. Under Private Network, select the Data Catalog is on Private Network checkbox if your data catalog is deployed in a private network environment that is not accessible from the public internet.

  5. Under OpenLineage Endpoint, enter the full URL where Alteryx should send OpenLineage event data.

  6. Under OpenLineage Namespace, enter the OpenLineage Job Namespace. Refer to your data catalog's documentation for details on namespace requirements.

  7. Under Auth Method, select your authentication method from the dropdown.

    Select either None or Bearer Token.

    • If you select None, no authentication credentials are passed when connecting to the data catalog. Use this option only if your catalog does not require authentication. If the catalog requires authentication, choosing None will result in connection failures.

    • If you select Bearer Token, enter the token—as created in your data catalog—into the Bearer Token field.

  8. Under Data Lineage Level, select Dataset or Column from the drop-down.

    Data Lineage Level defines how detailed you want lineage tracking to be.
This setting applies only to new runs. If the lineage service is down, Server uses the last known setting or defaults to dataset level. Once the service is restored, new runs follow your selected level.

    For more information on connection types, go to Supported Tools for Column-Level Lineage, section Supported Connection Types.

    • Select Dataset if you want to define the lineage at the dataset level only. Alteryx will not send mapping information.

      Dataset-level lineage shows how entire datasets (such as tables, files, or views) move and transform across systems. It provides a high-level overview of the flow of data.

    • Select Column if you want to define the lineage down to individual columns.

      Column-level lineage traces the journey of individual fields or attributes within a dataset. It explains exactly how each column was generated, transformed, or aggregated. For more information, go to Supported Tools for Column-Level Lineage.

  9. To test this connection, select Test Connection.

    Note

    If you have selected the Data Catalog is on Private Network checkbox in step 4, the Test Connection button is grayed out. To test your connection, go to Alteryx System Settings > Controller > Connections - Alteryx One. For more information, refer to Controller.

  10. Select Save.

Now that you have set up your data catalog connection. It will appear under the Data Consumer section.

Edit or Delete Data Catalog Connection

Go to the Data Consumer section and find the data connection you want to edit or delete. Then select View Details to see Data Catalog Connection Details.

To edit a data catalog connection, select Edit.

To delete a data catalog connection, select the 3-dot menu, then select Delete.

Data Source: Server

Once you set up your External Data Catalog, you can connect Alteryx One to your Server to transfer lineage metadata to the data catalog.

Set Up Server Connection

To send lineage metadata to your catalog, enable Server to connect to Alteryx One Workspace.

  1. In the menu, go to Admin Console > Data Lineage Connection.

  2. Under Data Source, select Set Up for Server.

  3. A Set Up Server page opens.

  4. Enter Server Name to help to identify it in this workspace.

  5. Now generate Server Link Payload. To do so, go to Create Server Link Payload and select Generate Payload. Copy and paste it into your Alteryx System Settings to complete the setup. For more information where to enter the Server Link Payload and how to test it in Server, go to Controller.

    The payload includes the client ID and client secret associated with the Service Principal.

  6. Select Done.

Now you have set up your Server connection.

Edit or Delete Server Connection

Go to the Data Source section and find the Server data connection you want to edit or delete. Then select View Details to see Server Connection Details.

To edit a data catalog connection, select Edit.

To delete a data catalog connection, select the 3-dot menu, then select Delete.

In this section: