File step

The File step writes a CSV or JSON file and uploads it to cloud storage or to your own FTP, FTPS or SFTP server. It can also fetch a file and read it into a variable. Use it for nightly exports, supplier feeds and anything else that's easier to hand over as a file.

Operations

Open, Append and Close let you stream big exports: open the file, let Fetch all pages write each page into it through its File handle, then close it.

Set it up

🚧

Keep logins in private data

Don't type passwords or keys into the step. Store them in private data, read them with an API Request (Get Private Data) and a Script step, and use {{ ftp_password }} in the field.

What it saves

• Write and Close save {destination, path, rows, bytes}, plus url for cloud storage or host for FTP, FTPS and SFTP.
• Fetch saves what it read: for CSV, an array with one object per row, keyed by the header (or one array per row with No header row); for JSON and NDJSON, the parsed data. With Auto, a .csv name is read as CSV, and anything starting with [ or { as JSON.

Limits

• Files can be up to 10 MB.
• Explode array path can produce up to 100,000 rows in one step. More fails with explode: more than 100000 rows in one step.
• A handle has to be opened before you append to or close it, and each handle can only be open once.

How retries work

Write and Close are the steps that publish the file. A top-level Write or Close that already succeeded isn't repeated when the job is retried, and nor are the steps that fed that file. Open and Append run again, and an upload that was never closed is thrown away rather than half-published.

Example 1: Export open order lines

Trigger: Scheduled.

1. Shopify GraphQL: Fetch every order page, with Fetch all pages on, Response variable orders (see Fetch all pages).
2. Transform Data: Keep the open lines, Source orders.items, with a Filter and a Map that builds order, sku and quantity. Response variable lines.
3. File: Write the export, Operation Write, Format CSV, Source lines, Columns order, sku, quantity, Destination Cloud storage, Filename open-lines.csv, Response variable export.
4. Send Notification: Send a Slack message with Open lines: {{ export.url }}.

Example 2: Send every product or sale to your FTP server

This is the nightly FTP export pattern: get the login from private data, open the file, stream every page into it, then close it.

Trigger: Scheduled.

1. API Request: Get Private Data, Key ftp_export, Response variable ftp_res. Store JSON like {"host": "ftp.example.com", "username": "export", "password": "…"} there.

2. Script: Build the request

   JavaScript

   var ftp = JSON.parse(JSON.parse($.ftp_res.body).ftp_export);
   $.ftp_host = ftp.host;
   $.ftp_user = ftp.username;
   $.ftp_password = ftp.password;
   $.filename = "sales-" + new Date().toISOString().slice(0, 10) + ".csv";

3. File: Open the export, Operation Open, Handle export, Format CSV, Columns id, receipt_number, store_id, total_price, Destination FTP (or FTPS or SFTP), Host {{ ftp_host }}, Username {{ ftp_user }}, Password {{ ftp_password }}, Filename {{ filename }}.

4. API Request: Fetch every page, endpoint List Sales (or Get Products for a product export), Fetch all pages on, File handle export. Each page is appended to the file as it arrives.

5. File: Close the export, Operation Close, Handle export, Response variable export_res.

6. If Condition: If the file has rows, Condition $.export_res.rows > 0, with a Send Notification under THEN.

To get one row per sale line, set Explode array path to lines on the fetch step, and use columns like id, lines.item_id and lines.quantity.

← All Glue step guides