Using Discovery APIs

By Lindsey Jenkins in GET/technical, API Pro Tips Posted Oct 13, 2020

You (and your users) can’t map and transform data if you’re missing custom objects.

Discovery APIs are critical to understanding a target application’s unique (often customized) data structure and, consequently, building effective and accurate mappings and transformations that ensure data sync. Seamless connectivity is only possible when you account for custom objects from the target application. 

This blog post is a follow-on to our updated Definitive Guide to API Integration. Download the full guide for more in-depth content on integration best practices, from pre-build to post-build, or check out the blog series.

discovery APIs

Your integration/application must discover the data structure of the target application and will need to present the discovered data structure, custom objects, and custom fields for you to evaluate and map. There are three key steps to do this successfully:

1. Identify Relevant Data Objects Within Your App

Are all of your application's data objects relevant to the data you’re targeting at the endpoint(s)? To determine the data you’re interested in integrating with, turn to your app’s data model to identify the standard objects at the endpoint that you will be integrating to your application. 

2. Discover Standard and Custom Objects

If you plan to support the integration and mapping of custom data, you’ll need an automated means to discover custom objects and fields at the endpoint. In addition to understanding the standard data objects, you’ll likely need to discover the data structure at the specific instance of the endpoint in the majority of SaaS application endpoints. While a standard object will persist across multiple instances of an integration endpoint, custom objects and data fields will only exist for a specific instance.

There’s two types of discovery mechanisms. Some endpoints such as Salesforce and Eloqua provide discovery APIs that you can use to programmatically discover all the fields contained within an object, giving you the ability to programmatically discover standard and custom objects and data fields. 

You’ll need to use these discovery APIs to find the equivalent objects in a cloud service you’re integrating to. By using a discovery API, you can discover all the fields related to the object you placed a discover API call against.

 

discovery APIs

Alternatively, if the endpoint doesn’t provide a Discovery API you will need to provide a mechanism to the user to retrieve the complete data structure of the object including custom fields. 

For RESTful endpoints you can issue a GET with the object ID for a fully populated custom object and the payload will return the complete list of fields. In some cases you can issue a GET with the object name to register a new custom object, then you can call the discovery API with the id to get a complete list of fields.

Developers that use Cloud Elements benefit from the discovery capabilities (among other premium features such as bulk, eventing, etc..) we offer out-of-the-box for our Catalog of 200+ Elements, even ones that don’t support the capabilities at the native endpoint - check it out if you’re curious. Onto the final step…

3. Start to Standardize on Patterns

Once you’ve collected enough data and evidence, you can start to figure out patterns your customers create. You can then use those patterns to automate the process. From those patterns you can essentially create standard API calls on custom objects.

If you can automate the discovery stage with really fast API calls, your app will surely be on the fast track toward a seamless experience. Think about it. The ultimate flow for an app is to click on a button, authenticate with my own credentials, use your recommendations for mapping my standard data to standard fields, and then use the pick-list to map custom data to custom fields.

Field Mappings for your end user should look like this:

Ready to learn more about discovery APIs and data mapping/transformation? Get the info you need in the Develop section of our updated Definitive Guide to API Integration.

Grab your copy

 

Even more? Read the next post in the series: API data mapping.