Skip to main content

Configuring Looker & LookML Connector

Now that you have created a DataHub-specific API key with the relevant access in the prior step, it's time to set up a connection via the DataHub UI.

Configure Secrets​

You must create two secrets to configure a connection with Looker or LookerML.

  • LOOKER_CLIENT_ID
  • LOOKER_CLIENT_SECRET

On your DataHub instance, navigate to the Ingestion tab in your screen's top right corner.

Navigate to the "Ingestion Tab"

note

If you do not see the Ingestion tab, please get in touch with your DataHub admin to grant you the correct permissions.

Navigate to the Secrets tab and click Create new secret.

Secrets Tab

First, create a secret for the Client Id. The value should be the Client Id of the API key created in the prior step.

API Key Client ID

Then, create a secret for the Client Secret. The value should be the Client Secret of the API key created in the prior step.

API Key client secret

Configure Looker Ingestion​

Configure Recipe​

Navigate to the Sources tab and click Create Source.

Click "Create Source"

Choose Looker.

Select Looker from the options

Enter a Source Name at the top of the Connection Details page, then enter the details into the Looker Recipe.

  • Base URL: This is your looker instance URL. (i.e. https://<your-looker-instance>.cloud.looker.com)
  • Client ID: Use the secret LOOKER_CLIENT_ID with the format ${LOOKER_CLIENT_ID}.
  • Client Secret: Use the secret LOOKER_CLIENT_SECRET with the format ${LOOKER_CLIENT_SECRET}.

Optionally, use the dashboard_pattern and chart_pattern fields to filter for specific dashboard and chart.

config:
...
dashboard_pattern:
allow:
- "2"
chart_pattern:
allow:
- "258829b1-82b1-4bdb-b9fb-6722c718bbd3"

Your recipe should look something like this:

Looker Recipe

Schedule Execution​

Now, it's time to schedule a recurring ingestion pipeline to extract metadata from your Looker instance regularly.

In the Sync Schedule section, turn on Run on a schedule, then decide how regularly you want this ingestion to run-- day, month, year, hour, minute, etc. Select from the dropdown.

schedule selector

Ensure you've configured your correct timezone.

timezone_selector

Save and Run​

Click Save and Run at the top of the page.

You will now find your new ingestion source running.

ingestion_running

Configure LookML Connector​

Now that you have created a DataHub-specific API key and Deploy Key with the relevant access in the prior step, it's time to set up a connection via the DataHub UI.

Configure Recipe​

Navigate to the Sources tab and click Create Source.

Click "Create Source"

Choose LooML.

Select Looker from the options

Enter a Source Name at the top of the Connection Details page, then enter the details into the Looker Recipe. You need to set a minimum 5 fields in the recipe for this quick ingestion guide:

  • GitHub Repository: This is your GitHub repository where LookML models are stored. You can provide the full URL (example: https://gitlab.com/gitlab-org/gitlab) or organization/repo; in this case, the connector assume it is a GitHub repo
  • GitHub Deploy Key: Copy the content of looker_datahub_deploy_key and paste into this filed.
  • Looker Base URL: This is your looker instance URL. (i.e. https://abc.cloud.looker.com)
  • Looker Client ID: Use the secret LOOKER_CLIENT_ID with the format ${LOOKER_CLIENT_ID}.
  • Looker Client Secret: Use the secret LOOKER_CLIENT_SECRET with the format ${LOOKER_CLIENT_SECRET}.

Your recipe should look something like this:

LookML Recipe

Schedule Execution​

Now, it's time to schedule a recurring ingestion pipeline to extract metadata from your Looker instance regularly.

In the Sync Schedule section, turn on Run on a schedule, then decide how regularly you want this ingestion to run-- day, month, year, hour, minute, etc. Select from the dropdown.

schedule selector

Ensure you've configured your correct timezone.

timezone_selector

Save and Run​

Click Save and Run at the top of the page.

You will now find your new ingestion source running.

ingestion_running

Validate Ingestion Runs​

View the latest status of ingestion runs on the Ingestion page.

ingestion succeeded

Open the Run History tab to see the complete list of historical runs and outcomes; click a run's status to open its Run Details page.

ingestion_details

In the Assets section of the Run Details page, click View all to see which entities were ingested.

ingestion_details_view_all

Pick an entity from the list to manually validate if it contains the detail you expected.

ingestion_details_view_all

Congratulations! You've successfully set up Looker & LookML as an ingestion source for DataHub!