Bulk import data with the GraphQL Admin API
Importing large volumes of data using traditional and synchronous APIs is slow, complex to run, and difficult to manage. Instead of manually running a GraphQL mutation multiple times and managing a client-side throttle, you can run a bulk mutation operation.
Using the GraphQL Admin API, you can bulk import large volumes of data asychronously. When the operation is complete, the results are delivered in a JSON Lines (JSONL) file that Shopify makes available at a URL.
This guide introduces the
bulkOperationRunMutation and shows you how to use it to bulk import data into Shopify.
- You're familiar with creating products, product variants, and collections in your development store.
- You're familiar with performing bulk operations using the GraphQL Admin API.
- Your API client has been granted the
bulk_graphql_mutation_apibeta flag from Shopify.
This limit is in place because operations are asynchronous and long-running. To run a subsequent bulk mutation operation for a shop, you need to either cancel the running operation or wait for it to finish.
You can supply only one of the supported GraphQL API mutations to the
bulkOperationRunMutationat a time:
The mutation that's passed into
bulkOperationRunMutationis limited to one connection field, which is defined by the GraphQL Admin API schema.
How bulk importing data works
You initiate a bulk operation by supplying a mutation string in the
bulkOperationRunMutation. Shopify then executes that mutation string asynchronously as a bulk operation.
Most GraphQL Admin API requests that you make are subject to rate limits, but the
bulkOperationRunMutation request isn't. Because you're only making low-cost requests for creating operations, polling their status, or canceling them, bulk mutation operations are an efficient way to create data compared to standard GraphQL API requests.
The following diagram shows the steps involved in bulk importing data into Shopify:
Create a JSONL file and include GraphQL variables: Include the variables for the mutation in a JSONL file format. Each line in the JSONL file represents one input unit. The mutation runs once on each line of the input file.
Upload the file to Shopify: Before you upload the file, you must reserve a link by running the
stagedUploadsCreatemutation. After the space has been reserved, you can upload the file by making a request using the information returned from the
Create a bulk mutation operation: After the file has been uploaded, you can run
bulkOperationRunMutationto create a bulk mutation operation. The
bulkOperationRunMutationimports data in bulk by running the supplied GraphQL API mutation with the file of variables uploaded in the last step.
Poll the status of the operation: While the operation is running, you need to poll to see its progress using the
objectCountfield on the
bulkOperationobject increments to indicate the operation's progress, and the
statusfield returns a boolean value that states whether the operation is completed.
Retrieve the results: When a bulk mutation operation is completed, a JSONL output file is available for download at the URL specified in the
Create a JSONL file and include GraphQL variables
When adding the required GraphQL variables to a new JSONL file, you need to format the variables so that they'll be accepted by the corresponding bulk operation GraphQL API.
For example, you might want to import a large quantity of products. Each attribute of a product must be mapped to existing fields defined in the GraphQL input object
ProductInput. In the JSONL file, each line represents one product input. The GraphQL Admin API runs once on each line of the input file. One input should take up one line only, no matter how complex the input object structure is.
The following example shows a sample JSONL file that is used to create 10 products in bulk: