Enhancing performance using Sugar 7 Bulk API

Matthew Marum —  July 20, 2015 — 6 Comments

If you’ve used Sugar 7.5 or earlier then you have probably noticed that it take a few moments after a record appears for all the subpanels to finish being populated with data.

Long list of subpanels

Long list of subpanels

This was because each subpanel generated a separate round-trip HTTP transaction from the browser to Sugar web server in order to fetch the related records for each represented relationship.

However, in Sugar 7.6 this behavior has been improved due to the introduction of the Bulk API.

Sugar 7 Bulk API

The Bulk API was introduced in Sugar 7.5, however the Sugar 7 client code did not start fully taking advantage of it for subpanels until the Sugar 7.6 release.  It exists at the /rest/v10/bulk endpoint.

For convenience, I’m including some of the in-app documentation for the Bulk API here but you can find more detailed API docs by visiting the /rest/v10/help endpoint on a Sugar 7.6 instance.

POST /bulk

Summary

This request will run a sequence of API requests within one query. The requests are executed sequentially and their results are returned as one response. Some requests may return failure code, that does not interrupt the execution of the batch, and the overall request will still be considered successful.

Request Parameters

Name Type Description Required
requests Array The list of requests True

Each of the requests can have the following fields:

Name Type Description Required
url String The request URL, starting with version. True
data JSON String The data for the POST/PUT body. Must be a JSON-encoded string. False
headers Array The request headers False
method String The HTTP method (default is GET) False

Response

The response will contain an array of response objects, each of them will correspond to the individual request. The following fields are in the response objects:

Name Type Description
contents Array or String The response contents, can be JSON object or string depending on what the individual request is supposed to return.
headers Array The response headers
status Integer HTTP status code of the response. Will be 2XX for successful requests and 4XX or 5XX for errors.

A Short Example

Imagine building an integration where you need to keep track of Accounts and Contacts that exist within the state of North Carolina.  This can be done with the v10 REST API today but it requires a separate Filter API request for each the Accounts and Contacts modules.  With the Bulk API, it is possible to combine the two requests into a single HTTP transaction.

POST /v10/rest/bulk

Request Body

{
  "requests": [
    {
      "url": "/v10/Accounts/filter/count",
      "method": "POST",
      "data": "{\"filter\":[{\"billing_address_state\":\"NC\"}]}"
    },
    {
      "url": "/v10/Contacts/filter/count",
      "method": "POST",
      "data": "{\"filter\":[{\"primary_address_state\":\"NC\"}]}"
    }
  ]
}

An important thing that should be pointed out for Bulk API requests.  When using the data property, the contents needs to be a JSON encoded string.  This means that JSON request bodies appear to be double encoded when used as part of a Bulk API sequence which explains the escaped quote marks in the example above.

When passing JSON data via Bulk API, this JSON data will need to contained within a JSON encoded string.

Response

[
  {
    "contents": {
      "record_count": "1"
    },
    "headers": [],
    "status": 200,
    "status_text": "OK"
  },
  {
    "contents": {
      "record_count": "1"
    },
    "headers": [],
    "status": 200,
    "status_text": "OK"
  }
]

When parsing result array from a Bulk API request, the actual response bodies are stored under the contents properties.  Also keep in mind the order requests are listed since each request is executed in turn.  If you make an update in the first request, later requests in the sequence will be affected by that update.

The Bulk API executes requests in array order sequence but returns all responses at once.

Taking advantage of Bulk API to improve performance

Using the Bulk API will have immediate beneficial impact for any long sequence of API calls.  It eliminates a lot of network roundtrip overhead that can be a real bottleneck for higher latency connections.  This benefit should be evident by the improved responsiveness of subpanels in Sugar 7.6.  But really where it shines is in clients or interfaces that need to move CRM data around in real time.

The Bulk API is the ideal solution when you need to move a lot of data in real time.

It is also easy to adopt as a standard gateway for the Sugar API since it can be used to execute a single request at a time.  So even if you don’t have a need for bulk transactions all the time, you can allow yourself flexibility to support them as needed.

Matthew Marum

Posts

Matt is the Director of Developer Advocacy for SugarCRM. Previously he was an Engineer on Sugar 7 and a Solutions Architect for the OEM program. He is also an avid trail runner, Boston Marathon qualifier and a karaoke aficionado.

6 responses to Enhancing performance using Sugar 7 Bulk API

  1. 

    Hi,
    I’m developping a Powershell script that updates at least 50 000 Accounts in SugarCRM every night. I’m using the Bulk API with packs of 500 Accounts.
    Does this API have a requests number limitation?

    Regards

    • 

      There are no limits on API requests with Sugar – whether it is deployed On-Site or running in Sugar OnDemand.

  2. 

    Sometimes response headers is an empty array and other times it is a JSON object with the headers as properties. Makes it a little tricky to handle generically.

  3. 

    With BULK api and 3 threads, I was able to load ~150,000 / hour on my Mac Book Pro 2015. This is definitely more efficient that single REST calls, as HTTP connections are the most expensive operations in terms of resource consumption.

  4. 

    Hi, I want to know, with document related that I need to know to modify or ijection if any changes data in sugarCRM will update using integration to another application. How to used that?

Trackbacks and Pingbacks:

  1. Migrating Salesforce documents to SugarCRM - MasterSolve Inc. - May 31, 2016

    […] the actual records into SugarCRM is well documented using either the import tool, or with web services, so we're going to skip that part of the process.  Below are the only fields we're going to need […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s