Eventing and Bulk with Cloud Elements

By Vivek Ravishankar in GET/technical Posted Oct 2, 2017

Integrations are all about enabling communication between your disconnected APIs. So, when building a new integration, one of your top priorities should be to create frameworks for data transfer. Specifically, methods for detecting events in your endpoints and for sending and receiving new data rapidly.

Luckily, we like to give your integrations super powers, so every single one of our elements comes with pre-built, standardized event handling and bulk action frameworks. If you use Cloud Elements, you can take advantage of these frameworks out of the box to get your integrations ready to go faster than ever.



Normally for detecting events happening within an endpoint, you have two options: webhooks or polling. For many reasons, which you can learn about here, webhooks tend to be the better option, and for vendors that directly support webhooks, you can set those up through Cloud Elements.

Salesforce, for example, has built-in functionality to post events to webhooks, and you can configure that functionality through the Cloud Elements UI and make sure that any data you receive is standardized. Unfortunately, there are still many API vendors that do not offer webhook functionality out of the box. For these APIs you’ll need to use a polling framework.

Polling vs Cloud Elements timeline.pngPolling works by querying the endpoint for any objects that have changed within a set polling interval. If the query finds anything, it will return the new or changed data. While this sounds like a simple process, successfully implementing polling can be very complicated and varies from vendor to vendor.

One of the great features of Cloud Elements is that all of our elements have a pre-built polling framework, so we handle the messy backend and automatically update any new data, just like using a webhook.

To set up a simple polling framework in Cloud Elements, all you have to do is set up the polling event type and provide some objects to monitor for changes. For example if you set up Cloud Elements to poll contacts every 15 minutes, that will say, “every 15 minutes, go out and check to see if any contacts have changed, and if that is the case, post the changes to a webhook.”



Cloud Elements has two ways of implementing bulk actions with our elements. For the endpoints that natively support batch or bulk APIs, we can take advantage of those directly, but many do not support this functionality. For these endpoints, we’ve built a custom framework on top of the standard API to sort of create fake bulk actions.

For example, if you want to pull all the contacts out of an element that doesn't have a native batch or bulk API, you can implement this add-on framework which will one-by-one, pull out all those contacts and push them into a single file that you can then download.

One thing to note about our bulk framework is that it works asynchronously. What that means is that in order to create a bulk download file, you’ll actually execute two API calls. The first one will be the POST /bulk/query call, which will create a bulk job and then pull, individually, all the contacts out of that endpoint. The second one you’ll use is the GET /bulk/{id}/{object name} , which will actually grab the downloaded file once that first bulk job is finished. We have a similar process for uploading in bulk to an endpoint as well.Screen Shot 2017-09-25 at 2.53.39 PM.png

For any of the endpoints that do not natively support bulk or batch, our bulk framework is definitely something you’ll want to know, since it is an enhancement on top of the base API that is only available through our platform. It gives you a big advantage over a standard DIY integration, because you can use it to sync first-time data or import existing data from one CRM system to another, so it’s definitely worth playing around with.

Haven't tried Cloud Elements yet? Request a free trial and check out how our catalog of over 200 standardized APIs can get your integrations up and running in no time.

Log in to Start Integrating

Learn how to authenticate in Cloud Elements v2 platform in this blog.