Bulk import data with the GraphQL Admin API

Importing large volumes of data using traditional and synchronous APIs is slow, complex to run, and difficult to manage. Instead of manually running a GraphQL mutation multiple times and managing a client-side throttle, you can run a bulk mutation operation.

Using the GraphQL Admin API, you can bulk import large volumes of data asychronously. When the operation is complete, the results are delivered in a JSON Lines (JSONL) file that Shopify makes available at a URL.

This guide introduces the bulkOperationRunMutation and shows you how to use it to bulk import data into Shopify.

Requirements

Limitations

How bulk importing data works

You initiate a bulk operation by supplying a mutation string in the bulkOperationRunMutation. Shopify then executes that mutation string asynchronously as a bulk operation.

Most GraphQL Admin API requests that you make are subject to rate limits, but the bulkOperationRunMutation request isn't. Because you're only making low-cost requests for creating operations, polling their status, or canceling them, bulk mutation operations are an efficient way to create data compared to standard GraphQL API requests.

The following diagram shows the steps involved in bulk importing data into Shopify:

Workflow for bulk importing data

  1. Create a JSONL file and include GraphQL variables: Include the variables for the mutation in a JSONL file format. Each line in the JSONL file represents one input unit. The mutation runs once on each line of the input file.

  2. Upload the file to Shopify: Before you upload the file, you must reserve a link by running the stagedUploadsCreate mutation. After the space has been reserved, you can upload the file by making a request using the information returned from the stagedUploadsCreate response.

  3. Create a bulk mutation operation: After the file has been uploaded, you can run bulkOperationRunMutation to create a bulk mutation operation. The bulkOperationRunMutation imports data in bulk by running the supplied GraphQL API mutation with the file of variables uploaded in the last step.

  4. Poll the status of the operation: While the operation is running, you need to poll to see its progress using the currentBulkOperation field. The objectCount field on the bulkOperation object increments to indicate the operation's progress, and the status field returns a boolean value that states whether the operation is completed.

  5. Retrieve the results: When a bulk mutation operation is completed, a JSONL output file is available for download at the URL specified in the url field.

Create a JSONL file and include GraphQL variables

When adding the required GraphQL variables to a new JSONL file, you need to format the variables so that they'll be accepted by the corresponding bulk operation GraphQL API.

For example, you might want to import a large quantity of products. Each attribute of a product must be mapped to existing fields defined in the GraphQL input object ProductInput. In the JSONL file, each line represents one product input. The GraphQL Admin API runs once on each line of the input file. One input should take up one line only, no matter how complex the input object structure is.

The following example shows a sample JSONL file that is used to create 10 products in bulk:

Upload