Importing large volumes of data using traditional and synchronous APIs is slow, complex to run, and difficult to manage. Instead of manually running a GraphQL mutation multiple times and managing a client-side throttle, you can run a bulk mutation operation.
Using the GraphQL Admin API, you can bulk import large volumes of data asychronously. When the operation is complete, the results are delivered in a [JSON Lines (JSONL)](https://jsonlines.org/) file that Shopify makes available at a URL.
This guide introduces the [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) and shows you how to use it to bulk import data into Shopify.
## Requirements
- You're familiar with creating [products](/docs/api/admin-graphql/latest/mutations/productcreate), [product variants](/docs/api/admin-graphql/latest/mutations/productvariantcreate), and [collections](/docs/api/admin-graphql/latest/mutations/collectioncreate) in your development store.
- You're familiar with [performing bulk operations](/docs/api/usage/bulk-operations/queries) using the GraphQL Admin API.
## Limitations
- You can run only one bulk operation of each type ([`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) or [`bulkOperationRunQuery`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunquery)) at a time per shop.
- The bulk mutation operation has to complete within 24 hours. After that it will be stopped and marked as `failed`.
When your import runs into this limit, consider reducing the input size.
- You can supply only one of the supported GraphQL API mutations to the `bulkOperationRunMutation` at a time:
- [`collectionCreate`](/docs/api/admin-graphql/latest/mutations/collectioncreate)
- [`collectionUpdate`](/docs/api/admin-graphql/latest/mutations/collectionupdate)
- [`customerCreate`](/docs/api/admin-graphql/latest/mutations/customercreate)
- [`customerUpdate`](/docs/api/admin-graphql/latest/mutations/customerupdate)
- [`customerPaymentMethodRemoteCreate`](/docs/api/admin-graphql/latest/mutations/customerpaymentmethodremotecreate)
- [`giftCardCreate`](/docs/api/admin-graphql/latest/mutations/giftcardcreate)
- [`giftCardUpdate`](/docs/api/admin-graphql/latest/mutations/giftcardupdate)
- [`marketingActivityUpsertExternal`](/docs/api/admin-graphql/latest/mutations/marketingActivityUpsertExternal)
- [`marketingEngagementCreate`](/docs/api/admin-graphql/latest/mutations/marketingEngagementCreate)
- [`metafieldsSet`](/docs/api/admin-graphql/latest/mutations/metafieldsset)
- [`metaobjectUpsert`](/docs/api/admin-graphql/latest/mutations/metaobjectupsert)
- [`priceListFixedPricesAdd`](/docs/api/admin-graphql/latest/mutations/pricelistfixedpricesadd)
- [`priceListFixedPricesDelete`](/docs/api/admin-graphql/latest/mutations/pricelistfixedpricesdelete)
- [`privateMetafieldUpsert`](/docs/api/admin-graphql/latest/mutations/privatemetafieldupsert)
- [`productCreate`](/docs/api/admin-graphql/latest/mutations/productcreate)
- [`productSet`](/docs/api/admin-graphql/latest/mutations/productSet)
- [`productUpdate`](/docs/api/admin-graphql/latest/mutations/productupdate)
- [`productUpdateMedia`](/docs/api/admin-graphql/latest/mutations/productupdatemedia)
- [`productVariantsBulkCreate`](/docs/api/admin-graphql/latest/mutations/productVariantsBulkCreate)
- [`productVariantsBulkDelete`](/docs/api/admin-graphql/latest/mutations/productVariantsBulkDelete)
- [`productVariantsBulkReorder`](/docs/api/admin-graphql/latest/mutations/productVariantsBulkReorder)
- [`productVariantsBulkUpdate`](/docs/api/admin-graphql/latest/mutations/productVariantsBulkUpdate)
- [`publishablePublish`](/docs/api/admin-graphql/latest/mutations/publishablePublish)
- [`publishableUnpublish`](/docs/api/admin-graphql/latest/mutations/publishableUnpublish)
- [`publicationUpdate`](/docs/api/admin-graphql/latest/mutations/publicationUpdate)
- [`storeCreditAccountCredit`](/docs/api/admin-graphql/latest/mutations/storecreditaccountcredit)
- [`subscriptionBillingAttemptCreate`](/docs/api/admin-graphql/latest/mutations/subscriptionbillingattemptcreate)
- [`subscriptionContractActivate`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractactivate)
- [`subscriptionContractAtomicCreate`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractatomiccreate)
- [`subscriptionContractCancel`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractcancel)
- [`subscriptionContractExpire`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractexpire)
- [`subscriptionContractFail`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractfail)
- [`subscriptionContractPause`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractpause)
- [`subscriptionContractProductChange`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractproductchange)
- [`subscriptionContractSetNextBillingDate`](/docs/api/admin-graphql/latest/mutations/subscriptioncontractsetnextbillingdate)
- The mutation that's passed into `bulkOperationRunMutation` is limited to one connection field, which is defined by the GraphQL Admin API schema.
- The size of the JSONL file cannot exceed 20MB.
## How bulk importing data works
You initiate a bulk operation by supplying a mutation string in the `bulkOperationRunMutation`. Shopify then executes that mutation string asynchronously as a bulk operation.
Most GraphQL Admin API requests that you make are subject to [rate limits](/docs/api/usage/rate-limits), but the `bulkOperationRunMutation` request isn't. Because you're only making low-cost requests for creating operations, polling their status, or canceling them, bulk mutation operations are an efficient way to create data compared to standard GraphQL API requests.
The following diagram shows the steps involved in bulk importing data into Shopify:
1. **Create a JSONL file and include GraphQL variables**: Include the variables for the mutation in a JSONL file format. Each line in the JSONL file represents one input unit. The mutation runs once on each line of the input file.
2. **Upload the file to Shopify**: Before you upload the file, you must reserve a link by running the [`stagedUploadsCreate`](/docs/api/admin-graphql/latest/mutations/stageduploadscreate) mutation. After the space has been reserved, you can upload the file by making a request using the information returned from the [`stagedUploadsCreate`](/docs/api/admin-graphql/latest/mutations/stageduploadscreate) response.
3. **Create a bulk mutation operation**: After the file has been uploaded, you can run [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) to create a bulk mutation operation. The [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) imports data in bulk by running the supplied GraphQL API mutation with the file of variables uploaded in the last step.
4. **Wait for the operation to finish**: To determine when the bulk mutation has finished, you can either:
1. **Subscribe to a webhook topic**: You can use the [`webhookSubscriptionCreate`](/docs/api/admin-graphql/latest/mutations/webhooksubscriptioncreate) mutation to subscribe to the `bulk_operations/finish` webhook topic in order to receive a webhook when any operation finishes - in other words, it has completed, failed, or been cancelled.
2. **Poll the status of the operation**: While the operation is running, you can poll to see its progress using the [`currentBulkOperation`](/docs/api/admin-graphql/latest/objects/queryroot) field. The `objectCount` field on the [`bulkOperation`](/docs/api/admin-graphql/latest/objects/bulkoperation) object increments to indicate the operation's progress, and the `status` field returns a boolean value that states whether the operation is completed.
5. **Retrieve the results**: When a bulk mutation operation is completed, a JSONL output file is available for download at the URL specified in the `url` field.
## Create a JSONL file and include GraphQL variables
When adding GraphQL variables to a new JSONL file, you need to format the variables so that they are accepted by the corresponding bulk operation GraphQL API. The format of the input variables need to match the GraphQL Admin API schema.
For example, you might want to import a large quantity of products. Each attribute of a product must be mapped to existing fields defined in the GraphQL input object [`ProductInput`](/docs/api/admin-graphql/latest/input-objects/productinput). In the JSONL file, each line represents one product input. The GraphQL Admin API runs once on each line of the input file. One input should take up one line only, no matter how complex the input object structure is.
The following example shows a sample JSONL file that is used to create 10 products in bulk:
```json
{ "input": { "title": "Sweet new snowboard 1", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 2", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 3", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 4", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 5", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 6", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 7", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 8", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 9", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 10", "productType": "Snowboard", "vendor": "JadedPixel" } }
```
> Note:
> The GraphQL Admin API doesn't serially process the contents of the JSONL file. Avoid relying on a particular sequence of lines and object order to achieve a desired result.
## Upload the file to Shopify
After you've created the JSONL file, and included the GraphQL variables, you can upload the file to Shopify. Before uploading the file, you need to first generate the upload URL and parameters.
### Generate the uploaded URL and parameters
You can use the [`stagedUploadsCreate`](/docs/api/admin-graphql/latest/mutations/stageduploadscreate) mutation to generate the values that you need to authenticate the upload. The mutation returns an array of [`stagedMediaUploadTarget`](/docs/api/admin-graphql/latest/objects/stagedmediauploadtarget) instances.
An instance of [`stagedMediaUploadTarget`](/docs/api/admin-graphql/latest/objects/stagedmediauploadtarget) has the following key properties:
- `parameters`: The parameters that you use to authenticate an upload request.
- `url`: The signed URL where you can upload the JSONL file that includes GraphQL variables.
The mutation accepts an input of type [`stagedUploadInput`](/docs/api/admin-graphql/latest/input-objects/stageduploadinput), which has the following fields:
| Field | Type | Description |
|---|---|---|
| `resource` | [enum](/docs/api/admin-graphql/latest/enums/stageduploadtargetgenerateuploadresource) | Specifies the resource type to upload. To use `bulkOperationRunMutation`, the resource type must be `BULK_MUTATION_VARIABLES`. |
| `filename` | [string](/docs/api/admin-graphql/latest/scalars/String) | The name of the file to upload. |
| `mimeType` | [string](/docs/api/admin-graphql/latest/scalars/String) | The [media type](https://en.wikipedia.org/wiki/Media_type) of the file to upload. To use `bulkOperationRunMutation`, the `mimeType` must be `"text/jsonl"`. |
| `httpMethod` | [enum](/docs/api/admin-graphql/latest/enums/stageduploadhttpmethodtype) | The HTTP method to be used by the staged upload. To use `bulkOperationRunMutation`, the `httpMethod` must be `POST`. |
#### Example
The following example uses the [`stagedUploadsCreate`](/docs/api/admin-graphql/latest/mutations/stageduploadscreate) mutation to generate the values required to upload a JSONL file and be consumed by the [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation). You must first run the `stagedUploadsCreate` mutation with no variables, and then separately send a POST request to the staged upload URL with the JSONL data:
### Upload the JSONL file
After you generate the parameters and URL for an upload, you can upload the JSONL file using a POST request. You must use a multipart form, and include all parameters as form inputs in the request body.
To generate the parameters for the multipart form, start with the parameters returned from the `stagedUploadsCreate` mutation. Then, add the file attachment.
> Note:
> The `file` parameter must be the last parameter in the list. If you add the `file` parameter somewhere else, then you'll receive an error.
**POST request**
```curl
curl --location --request POST 'https://shopify-staged-uploads.storage.googleapis.com/' \
--form 'key="tmp/21759409/bulk/2d278b12-d153-4667-a05c-a5d8181623de/bulk_op_vars"' \
--form 'x-goog-credential="merchant-assets@shopify-tiers.iam.gserviceaccount.com/20220830/auto/storage/goog4_request"' \
--form 'x-goog-algorithm="GOOG4-RSA-SHA256"' \
--form 'x-goog-date="20220830T025127Z"' \
--form 'x-goog-signature="4c0f6920cd67cbdf1faae75c112a98d49f9751e4e0c9f525c850f15f40629afa13584ab9937ec9f5292065ca8fd357ba87e98d6ab0e383e15a6e444c7e9bae06fb95dc422ad673fe77aefcb68e9d1a6d55deb478e6976b61769e20863992fffd4036898f76c7a50e92f18aa4d9e3e04aa8d04086386dc0e488f2ccb0ebcc30c17da2ba5a4d6a9cd57553b41ef6698dbefc78a9b3fe1af167ea539b70e83e5fb015f061399e952270202b769ae8f4e7e50e97dbe6679c3281ec3fc886c3a67becc7b3cee1d0e6a2d0777d09f6d7b083499c58f9c566eeb5374afd67e26c7ab2a91cfe5c5deb83a507d7e3c3ea44bb9f401afd3f2e6b09742baff2b30bc3def78a"' \
--form 'policy="zyJjb25kaXRpb25zIjpbeyJDb250ZW50LVR5cGUiOiJ0ZXh0XC9qc29ubCJ9LHsic3VjY2Vzc19hY3Rpb25fc3RhdHVzIjoiMjAxIn0seyJhY2wiOiJwcml2YXRlIn0sWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsMSwyMDk3MTUyMF0seyJidWNrZXQiOiJzaG9waWZ5LXN0YWdlZC11cGxvYWRzIn0seyJrZXkiOiJ0bXBcL2djc1wvMTQzMjMyMjEwNFwvYnVsa1wvM2QyNzhiMTItZDE1My00NjY3LWEwNWMtYTVkODE4MTYyM2RlXC9idWxrX29wX3ZhcnMifSx7IngtZ29vZy1kYXRlIjoiMjAyMjA4MzBUMDI1MTI3WiJ9LHsieC1nb29nLWNyZWRlbnRpYWwiOiJtZXJjaGFudC1hc3NldHNAc2hvcGlmeS10aWVycy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbVwvMjAyMjA4MzBcL2F1dG9cL3N0b3JhZ2VcL2dvb2c0X3JlcXVlc3QifSx7IngtZ29vZy1hbGdvcml0aG0iOiJHT09HNC1SU0EtU0hBMjU2In1dLCJleHBpcmF0aW9uIjoiMjAyMi0wOC0zMVQwMjo1MToyN1oifQ=="' \
--form 'acl="private"' \
--form 'Content-Type="text/jsonl"' \
--form 'success_action_status="201"' \
--form 'file=@"/Users/username/Documents/bulk_mutation_tests/products_long.jsonl"'
```
**GraphQL variables in JSONL file**
```json
{ "input": { "title": "Sweet new snowboard 1", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 2", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 3", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 4", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 5", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 6", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 7", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 8", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 9", "productType": "Snowboard", "vendor": "JadedPixel" } }
{ "input": { "title": "Sweet new snowboard 10", "productType": "Snowboard", "vendor": "JadedPixel" } }
```
## Create a bulk mutation operation
After you upload the file, you can run [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) to import data in bulk. You must supply the corresponding mutation and the URL that you obtained in the [previous step](#generate-the-uploaded-url-and-parameters).
The [`bulkOperationRunMutation`](/docs/api/admin-graphql/latest/mutations/bulkoperationrunmutation) mutation takes the following arguments:
| Field | Type | Description |
|---|---|---|
| `mutation` | [string](/docs/api/admin-graphql/latest/scalars/String) | Specifies the GraphQL API mutation that you want to run in bulk. Valid values: [`productCreate`](/docs/api/admin-graphql/latest/mutations/productcreate), [`collectionCreate`](/docs/api/admin-graphql/latest/mutations/collectioncreate), [`productUpdate`](/docs/api/admin-graphql/latest/mutations/productupdate), [`productUpdateMedia`](/docs/api/admin-graphql/latest/mutations/productupdatemedia) |
| `stagedUploadPath` | [string](/docs/api/admin-graphql/latest/scalars/String) | The path to the file of inputs in JSONL format to be consumed by [`stagedUploadsCreate`](/docs/api/admin-graphql/latest/mutations/stageduploadscreate) |
### Example
In the following example, you want to run the following [`productCreate`](/docs/api/admin-graphql/latest/mutations/productcreate) mutation in bulk:
To run the `productCreate` mutation in bulk, pass the mutation as a string into `bulkOperationRunMutation`:
## Wait for the operation to finish
> Tip:
> Subscribing to the webhook topic is recommended over polling as it limits the number of redundant API calls.
### Option A. Subscribe to the `bulk_operations/finish` webhook topic
> Note:
> Using webhooks with bulk operations is only available in Admin API version 2021-10 and higher.
You can use the [`webhookSubscriptionCreate`](/docs/api/admin-graphql/latest/mutations/webhooksubscriptioncreate) mutation to subscribe to the `bulk_operations/finish` webhook topic in order to receive a webhook when any operation finishes - in other words, it has completed, failed, or been cancelled.
For full setup instructions, refer to [Configuring webhooks](/docs/apps/build/webhooks/subscribe).
After you've subscribed to the webhook topic, Shopify sends a POST request to the specified URL any time a bulk operation on the store (both mutations and [queries](/docs/api/usage/bulk-operations/queries)) finishes.
**Example webhook response**
```json
{
"admin_graphql_api_id": "gid://shopify/BulkOperation/206005076024",
"completed_at": "2024-01-28T19:11:09Z",
"created_at": "2024-01-28T19:10:59Z",
"error_code": null,
"status": "completed",
"type": "mutation",
}
```
You now must retrieve the bulk operation's data URL by using the `node` field and passing the `admin_graphql_api_id` value from the webhook payload as its `id`:
For more information on how webhooks work, refer to [Webhooks](/docs/apps/build/webhooks).
> Note:
> Webhook delivery isn't always guaranteed, so you might still need to poll for the operation's status to check when it's finished.
### Option B. Poll the status of the operation
While the operation is running, you can poll to see its progress using the [`currentBulkOperation`](/docs/api/admin-graphql/latest/objects/queryroot) field. The `objectCount` field increments to indicate the operation's progress, and the `status` field returns whether the operation is completed.
You can adjust your polling intervals based on the amount of data that you import. To learn about other possible operation statuses, refer to the [`BulkOperationStatus`](/docs/api/admin-graphql/latest/enums/bulkoperationstatus) reference documentation.
To poll the status of the operation, use the following example request:
## Retrieve the results
When a bulk mutation operation is finished, you can download a result data file.
If an operation successfully completes, then the `url` field contains a URL where you can download the data file. If an operation fails, but some data was retrieved before the failure occurred, then a partially complete data file is available at the URL specified in the `partialDataUrl` field.
In either case, the returned URLs are authenticated and expire after one week.
After you've downloaded the data, you can parse it according to the JSONL format. Since both input and response files are in JSONL, each line in the final asset file represents the response of running the mutation on the corresponding line in the input file.
### Operation success
The following example shows the response for a product that was successfully created:
### Operation failures
Bulk operations can fail for any of the reasons that a regular GraphQL API mutation would fail, such as not having permission to access certain APIs. For this reason, the best approach is to run a single GraphQL mutation first to make sure that it works before running a mutation as part of a bulk operation.
If a bulk operation does fail, then its `status` field returns `FAILED` and the `errorCode` field returns a code such as one of the following:
- `ACCESS_DENIED`: There are missing access scopes. Run the mutation normally (outside of a bulk operation) to get more details on which field is causing the issue.
- `INTERNAL_SERVER_ERROR`: Something went wrong on Shopify's server and we've been notified of the error. These errors might be intermittent, so you can try making your request again.
- `TIMEOUT`: One or more mutation timeouts occurred during execution. Try removing some fields from your query so that it can run successfully. These timeouts might be intermittent, so you can try submitting the query again.
To learn about the other possible operation error codes, refer to the [`BulkOperationErrorCode`](/docs/api/admin-graphql/latest/enums/BulkOperationErrorCode) reference documentation.
> Note:
> When using the `bulk_operations/finish` webhook, the `error_code` and `status` fields in the webhook payload will be lowercase. For example, `failed` instead of `FAILED`.
#### Validation error
If the input has the correct format, but one or more values failed the validation of the product creation service, then the response looks like the following:
```json
{"data"=>{"productCreate"=>{"userErrors"=>[{"message"=>"Some error message", "field"=>["some field"]}]}}}
```
#### Unrecognizable field error
If the input has an unrecognizable field, then the response looks like the following:
```json
{"errors"=>[{"message"=>"Variable input of type ProductInput! was provided invalid value for myfavoriteaddress (Field is not defined on ProductInput)", "locations"=>[{"line"=>1, "column"=>13}], "extensions"=>{"value"=>{"myfavoriteaddress"=>"test1"}, "problems"=>[{"path"=>["myfavoriteaddress"], "explanation"=>"Field is not defined on ProductInput"}]}}]}
```
> Note:
> This check is executed by comparing the input against the [`productInput`](/docs/api/admin-graphql/latest/input-objects/productinput) object, which is specified as part of the mutation argument.
## Cancel an operation
To cancel an in-progress bulk operation, run the [`bulkOperationCancel`](/docs/api/admin-graphql/latest/mutations/bulkoperationcancel) mutation and supply the operation ID as an input variable:
## Next steps
- Consult our [reference documentation](/docs/api/admin-graphql/latest/objects/BulkOperation) to learn more about creating and managing bulk operations.
- Learn how use bulk operations to [asynchronously fetch data in bulk](/docs/api/usage/bulk-operations/queries).