Overview and authentication

Install the sqlflow command-line tool, point it at your control plane, sign in, and learn its options, variables and exit codes.

sqlflow is a command-line tool for managing SQL Flow from a terminal or a CI/CD job. It lists and approves agents, saves connections, and creates, starts, pauses, reloads and deletes pipelines. It works with the same control plane as the web app, so whatever you create with it appears in the web app, and the other way around. Data never passes through it.

Before you begin

• The sqlflow executable on the machine where you run commands.
• A SQL Flow user who belongs to an account, and that user's password. Create users under Accounts. See Accounts and team.
• The control plane's https:// address.
• Outbound HTTPS from the machine to the control plane.

Install the CLI

sqlflow is a single executable, sqlflow.exe for Windows x64 and sqlflow for Linux x64.

1. In the web app, open Downloads. On the SQL Flow CLI card, select Windows x64 or Linux x64. Check the file's SHA-256 checksum under Details; see Verify a download.
2. Copy the file to a folder on your PATH. On Linux, make it executable with chmod +x sqlflow.
3. Under Get started with the CLI on Downloads, choose Windows x64 or Linux x64. It shows the commands that sign in and list your agents, with your control plane's address and your email already filled in.

Check that it starts. sqlflow --version prints the release version, and sqlflow with no arguments prints its usage.

sqlflow --version
sqlflow

Set the control plane address and your sign-in

Set these three environment variables before you run any command that talks to the control plane:

$env:SQLFLOW_URL = 'https://<control_plane>'
$env:SQLFLOW_OPERATOR = '<email>'
$secure = Read-Host -AsSecureString -Prompt 'SQL Flow password'
$env:SQLFLOW_OPERATOR_PASSWORD = [System.Net.NetworkCredential]::new('', $secure).Password

Replace <control_plane> and <email> with your values. These variables last for the current shell. In a CI/CD job, set them from your secret store, and see Idempotency keys and CI/CD.

A flag overrides its variable, and can go anywhere on the command line. Prefer the variables to --operator-password: a password on the command line is visible in the process list and your shell history.

Check the setup:

sqlflow agents

A table of agents, or a message that none are registered yet, means the CLI reached the control plane and signed in.

How the CLI signs in

• As a user. The CLI signs in with an email and password, the same as the web app.
• Once per command. Each command signs in, does its work and exits. The CLI keeps no session on disk.
• With that user's role. Commands that only read, such as agents, connection list and pipeline status, work for every role. Commands that change something need the User or Account admin role. A Viewer is refused with You don't have permission to perform this action. See Accounts and team.
• In one account. The CLI works in the first account the user was added to, and can't switch to another.
• After the first password change. A user with a temporary password must choose their own password in the web app first.

Each sign-in counts against the control plane's limit of 10 sign-ins a minute from one client address, so a script that runs many commands in a minute can be refused. See Idempotency keys and CI/CD.

HTTPS and certificates

The control plane accepts connections over HTTPS only, so --url must start with https://. The CLI checks the control plane's certificate against your operating system's trust store.

If your control plane's certificate comes from a certificate authority your operating system already trusts, as the hosted control plane's does, nothing more is needed. If a control plane you run yourself issues its own certificate, add its certificate authority to the trust store first. Get the certificate from your administrator, or open https://<control_plane>/ca.pem and save it, then compare its SHA-256 fingerprint with the one your administrator gives you before you trust it.

certutil -addstore -f Root .\ca.pem

Argument order

Write the two command words first, then the name the command acts on, then the command's flags. --url, --operator-user and --operator-password can go anywhere on the line, including before the command words:

sqlflow --url https://sqlflow.example.com pipeline pause orders-sql-to-pg
sqlflow pipeline pause orders-sql-to-pg --url https://sqlflow.example.com

A command flag between the verb and the name makes the CLI report that it needs a name. A flag takes the word after it as its value, so write --tables a,b, never --tables=a,b.

Commands

Output and exit codes

Results go to standard output. Errors go to standard error as error: <message>. When the control plane refuses a request that changes something, the message carries its status and its reason, for example error: Control plane archive pipeline orders failed (409 Conflict): <reason> [<code>]. A refused request that carries a database password (pipeline create, provision, schema-mapping and set-connection) shows the status and code only, so a credential is never echoed. A refused read request shows only the status, such as error: Response status code does not indicate success: 403 (Forbidden).

Each request waits up to 30 seconds for an answer.

Troubleshooting

Unknown command

error: unknown command. Run sqlflow without arguments for usage.

Cause: the command words are misspelled or missing, or a command flag comes before them. Fix: check the command against the usage the CLI prints. See Argument order.

Operator username not set

error: Operator username is required: set --operator-user or SQLFLOW_OPERATOR.

Cause: the CLI has no email to sign in with. The same check for the password says Operator password is required. Fix: set SQLFLOW_OPERATOR and SQLFLOW_OPERATOR_PASSWORD. See Set the control plane address and your sign-in.

Control plane login fails with 401

error: Control plane login failed (401). Check the operator credentials.

Cause: the email or password is wrong, or the user belongs to no account. Fix: check SQLFLOW_OPERATOR and SQLFLOW_OPERATOR_PASSWORD, and sign in to the web app with the same details.

Flag removed in an earlier release

error: --dest was removed; use --target.

Cause: the command uses a flag from an earlier release. Fix: use the flag the message names. The target flags are --target, --target-password-env, --target-password and --target-secret-ref.

Password change required

Choose a new password before continuing. [password_change_required]

Cause: the user still has a temporary password. A command that only reads shows error: Response status code does not indicate success: 403 (Forbidden). instead. Fix: sign in to the web app, choose your own password, and run the command again.

Too many sign-in requests

Too many requests from this client; retry later. [rate_limited]

Cause: more than 10 sign-ins in a minute from your address. Fix: wait a minute, and run fewer commands per minute.

Control plane certificate not trusted

error: The SSL connection could not be established

Cause: the operating system doesn't trust the control plane's certificate. Fix: add the control plane's certificate authority to the trust store. See HTTPS and certificates.

Permission denied

You don't have permission to perform this action.

Cause: your user has the Viewer role and the command changes something. Fix: use a user with the User or Account admin role.

See also

• Agents commands
• Pipelines commands
• Idempotency keys and CI/CD