Sep 17, 2024 Update: Batch Requests have been deprecated. Please use the Bulk API instead for the same purpose.
For those of you with heavy call volume, our Batch API lets you submit up to 50 API calls in a single request, and set a custom timeout parameter to make sure you get what you want on your own timeline.
Full documentation is in our Developer Dashboard. Here’s a quick introduction:
Package each individual API call in its own JSON object, comprised of:
- Method (GET or POST)
- The URL to the specific API you want to call — including all individual API parameters, even your token — minus “https://www.diffbot.com”
Here’s a sample object that calls our Article API. (Note that the URL is URL-encoded.)
{"method": "GET", "relative_url": "/api/article?url=http%3A%2F%2Fblogs.wsj.com%2Fventurecapital%2F2012%2F05%2F31%2Finvestors-back-diffbots-visual-learning-robot-for-web-content%2F%3Fmod%3Dgoogle_news_blog%26token=<token>"}
Post the JSON objects to https://www.diffbot.com/api/batch, along with your developer token.
Here’s an example using curl that makes two requests of our Article API:
curl -d 'token=<token>' -d 'batch=[{"method": "GET", "relative_url": "/api/article?url=http%3A%2F%2Fblogs.wsj.com%2Fventurecapital%2F2012%2F05%2F31%2Finvestors-back-diffbots-visual-learning-robot-for-web-content%2F%3Fmod%3Dgoogle_news_blog%26token=<token>"},{"method": "GET", "relative_url": "/api/article?url=http%3A%2F%2Fgigaom.com%2Fcloud%2Fsilicon-valley-royalty-pony-up-2m-to-scale-diffbots-visual-learning-robot%2F%26token=<token>"}]' https://www.diffbot.com/api/batch
Optionally, include a timeout parameter in your POST
Specify in milliseconds how long you want to wait for results. The Batch API will return whatever it’s able to retrieve within that timeframe.
If you don’t submit a “timeout” parameter, the API will return its results when all submitted URLs have received individual responses.
Happy batching!