Skip to content

Commit

Permalink
feat: remove oauth-config support
Browse files Browse the repository at this point in the history
  • Loading branch information
Zane Clark committed Nov 8, 2024
1 parent db86db4 commit 1beb3a5
Show file tree
Hide file tree
Showing 14 changed files with 11 additions and 223 deletions.
55 changes: 10 additions & 45 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -331,28 +331,12 @@ token will need to be supplied or acquired in one of the following ways:
1. The `--snowflake-token-path` [command-line argument](#commands)
2. Setting a `snowflake-token-path` in the [schemachange-config.yml](#yaml-config-file) file
3. Setting a `token_file_path` in the [connections.toml](#connectionstoml-file) file
3. Supply an "OAuth config" in one of the following ways (in order of priority):

1. The `--oauth-config` [command-line argument](#commands)
2. Setting an `oauthconfig` in the [schemachange-config.yml](#yaml-config-file) file

Since different Oauth providers may require different information the Oauth
configuration uses four named variables that are fed into a POST request to obtain a token. Azure is shown in the
example YAML but other providers should use a similar pattern and request payload contents.

* token-provider-url
The URL of the authenticator resource that will receive the POST request.
* token-response-name
The Expected name of the JSON element containing the Token in the return response from the authenticator resource.
* token-request-payload
The Set of variables passed as a dictionary to the `data` element of the request.
* token-request-headers
The Set of variables passed as a dictionary to the `headers` element of the request.

It is recommended to use the YAML file and pass oauth secrets into the configuration using the templating engine
instead of the command line option.

The OAuth POST call will only be made if a token or token filepath isn't discovered.
**Schemachange no longer supports the `--oauth-config` option.** Prior to the 4.0 release, this library supported
supplying an `--oauth-config` that would be used to fetch an OAuth token via the `requests` library. This required
Schemachange to keep track of connection arguments that could otherwise be passed directly to the Snowflake Python
connector. Maintaining this logic in Schemachange added unnecessary complication to the repo and prevented access to
recent connector parameterization features offered by the Snowflake connector.

### External Browser Authentication

Expand Down Expand Up @@ -460,13 +444,13 @@ snowflake-schema: null
# The Snowflake Authenticator to use. One of snowflake, oauth, externalbrowser, or https://<okta_account_name>.okta.com
snowflake-authenticator: null
# Path to file containing private key.
# Path to file containing private key.
snowflake-private-key-path: null
# Path to the file containing the OAuth token to be used when authenticating with Snowflake.
snowflake-token-path: null
# Override the default connections.toml file path at snowflake.connector.constants.CONNECTIONS_FILE (OS specific)
# Override the default connections.toml file path at snowflake.connector.constants.CONNECTIONS_FILE (OS specific)
connections-file-path: null
# Override the default connections.toml connection name. Other connection-related values will override these connection values.
Expand Down Expand Up @@ -496,24 +480,6 @@ dry-run: false
# A string to include in the QUERY_TAG that is attached to every SQL statement executed
query-tag: 'QUERY_TAG'
# Information for Oauth token requests
oauth-config:
# url Where token request are posted to
token-provider-url: 'https://login.microsoftonline.com/{{ env_var('AZURE_ORG_GUID', 'default') }}/oauth2/v2.0/token'
# name of Json entity returned by request
token-response-name: 'access_token'
# Headers needed for successful post or other security markings ( multiple labeled items permitted
token-request-headers:
Content-Type: "application/x-www-form-urlencoded"
User-Agent: "python/schemachange"
# Request Payload for Token (it is recommended pass
token-request-payload:
client_id: '{{ env_var('CLIENT_ID', 'default') }}'
username: '{{ env_var('USER_ID', 'default') }}'
password: '{{ env_var('USER_PASSWORD', 'default') }}'
grant_type: 'password'
scope: '{{ env_var('SESSION_SCOPE', 'default') }}'
```

#### Yaml Jinja support
Expand Down Expand Up @@ -610,7 +576,6 @@ usage: schemachange deploy [-h] [--config-folder CONFIG_FOLDER] [--config-file-n
| -v, --verbose | Display verbose debugging details during execution. The default is 'False'. |
| --dry-run | Run schemachange in dry run mode. The default is 'False'. |
| --query-tag | A string to include in the QUERY_TAG that is attached to every SQL statement executed. |
| --oauth-config | Define values for the variables to Make Oauth Token requests (e.g. {"token-provider-url": "https//...", "token-request-payload": {"client_id": "GUID_xyz",...},... })' |

### render

Expand Down Expand Up @@ -657,13 +622,13 @@ schemachange is a single python script located at [schemachange/cli.py](schemach
follows:

```
python schemachange/cli.py [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG] [--oauth-config OUATH_CONFIG]
python schemachange/cli.py [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG]
```

Or if installed via `pip`, it can be executed as follows:

```
schemachange [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG] [--oauth-config OUATH_CONFIG]
schemachange [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG]
```

The [demo](demo) folder in this project repository contains three schemachange demo projects for you to try out. These
Expand Down Expand Up @@ -702,7 +667,7 @@ If your build agent has a recent version of python 3 installed, the script can b

```bash
pip install schemachange --upgrade
schemachange [-h] [-f ROOT_FOLDER] -a SNOWFLAKE_ACCOUNT -u SNOWFLAKE_USER -r SNOWFLAKE_ROLE -w SNOWFLAKE_WAREHOUSE [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG] [--oauth-config OUATH_CONFIG]
schemachange [-h] [-f ROOT_FOLDER] -a SNOWFLAKE_ACCOUNT -u SNOWFLAKE_USER -r SNOWFLAKE_ROLE -w SNOWFLAKE_WAREHOUSE [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG]
```

Or if you prefer docker, set the environment variables and run like so:
Expand Down
10 changes: 0 additions & 10 deletions schemachange/config/DeployConfig.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
from schemachange.config.utils import (
get_snowflake_identifier_string,
validate_file_path,
get_oauth_token,
)


Expand Down Expand Up @@ -70,20 +69,11 @@ def factory(
# If set by an environment variable, pop snowflake_token_path from kwargs
if "snowflake_oauth_token" in kwargs:
kwargs.pop("snowflake_token_path", None)
kwargs.pop("oauth_config", None)
# Load it from a file, if provided
elif "snowflake_token_path" in kwargs:
kwargs.pop("oauth_config", None)
oauth_token_path = kwargs.pop("snowflake_token_path")
with open(oauth_token_path) as f:
kwargs["snowflake_oauth_token"] = f.read()
# Make the oauth call if authenticator == "oauth"

elif "oauth_config" in kwargs:
oauth_config = kwargs.pop("oauth_config")
authenticator = kwargs.get("snowflake_authenticator")
if authenticator is not None and authenticator.lower() == "oauth":
kwargs["snowflake_oauth_token"] = get_oauth_token(oauth_config)

change_history_table = ChangeHistoryTable.from_str(
table_str=change_history_table
Expand Down
6 changes: 1 addition & 5 deletions schemachange/config/get_merged_config.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,9 @@


def get_yaml_config_kwargs(config_file_path: Optional[Path]) -> dict:
# TODO: I think the configuration key for oauthconfig should be oauth-config.
# This looks like a bug in the current state of the repo to me

# load YAML inputs and convert kebabs to snakes
kwargs = {
k.replace("-", "_").replace("oauthconfig", "oauth_config"): v
for (k, v) in load_yaml_config(config_file_path).items()
k.replace("-", "_"): v for (k, v) in load_yaml_config(config_file_path).items()
}

if "verbose" in kwargs:
Expand Down
8 changes: 0 additions & 8 deletions schemachange/config/parse_cli_args.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -215,14 +215,6 @@ def parse_cli_args(args) -> dict:
help="The string to add to the Snowflake QUERY_TAG session value for each query executed",
required=False,
)
parser_deploy.add_argument(
"--oauth-config",
type=json.loads,
help='Define values for the variables to Make Oauth Token requests (e.g. {"token-provider-url": '
'"https//...", "token-request-payload": {"client_id": "GUID_xyz",...},... })',
required=False,
)

parser_render = subcommands.add_parser(
"render",
description="Renders a script to the console, used to check and verify jinja output from scripts.",
Expand Down
22 changes: 0 additions & 22 deletions schemachange/config/utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@
from pathlib import Path
from typing import Any

import json

import requests
import jinja2
import jinja2.ext
import structlog
Expand Down Expand Up @@ -220,23 +218,3 @@ def get_env_kwargs() -> dict[str, str]:
"connection_name": os.getenv("SNOWFLAKE_DEFAULT_CONNECTION_NAME"),
}
return {k: v for k, v in env_kwargs.items() if v is not None}


def get_oauth_token(oauth_config: dict):
req_info = {
"url": oauth_config["token-provider-url"],
"headers": oauth_config["token-request-headers"],
"data": oauth_config["token-request-payload"],
}
token_name = oauth_config["token-response-name"]
response = requests.post(**req_info)
response_dict = json.loads(response.text)
try:
return response_dict[token_name]
except KeyError:
keys = ", ".join(response_dict.keys())
errormessage = f"Response Json contains keys: {keys} \n but not {token_name}"
# if there is an error passed with the response include that
if "error_description" in response_dict.keys():
errormessage = f"{errormessage}\n error description: {response_dict['error_description']}"
raise KeyError(errormessage)
12 changes: 0 additions & 12 deletions tests/config/schemachange-config-full-no-connection.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,3 @@ autocommit: false
verbose: false
dry-run: false
query-tag: 'query-tag-from-yaml'
oauthconfig:
token-provider-url: 'token-provider-url-from-yaml'
token-response-name: 'token-response-name-from-yaml'
token-request-headers:
Content-Type: 'Content-Type-from-yaml'
User-Agent: 'User-Agent-from-yaml'
token-request-payload:
client_id: 'id-from-yaml'
username: 'username-from-yaml'
password: 'password-from-yaml'
grant_type: 'type-from-yaml'
scope: 'scope-from-yaml'
12 changes: 0 additions & 12 deletions tests/config/schemachange-config-full.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,3 @@ autocommit: false
verbose: false
dry-run: false
query-tag: 'query-tag-from-yaml'
oauthconfig:
token-provider-url: 'token-provider-url-from-yaml'
token-response-name: 'token-response-name-from-yaml'
token-request-headers:
Content-Type: 'Content-Type-from-yaml'
User-Agent: 'User-Agent-from-yaml'
token-request-payload:
client_id: 'id-from-yaml'
username: 'username-from-yaml'
password: 'password-from-yaml'
grant_type: 'type-from-yaml'
scope: 'scope-from-yaml'
12 changes: 0 additions & 12 deletions tests/config/schemachange-config-partial-with-connection.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,3 @@ autocommit: false
verbose: false
dry-run: false
query-tag: 'query-tag-from-yaml'
oauthconfig:
token-provider-url: 'token-provider-url-from-yaml'
token-response-name: 'token-response-name-from-yaml'
token-request-headers:
Content-Type: 'Content-Type-from-yaml'
User-Agent: 'User-Agent-from-yaml'
token-request-payload:
client_id: 'id-from-yaml'
username: 'username-from-yaml'
password: 'password-from-yaml'
grant_type: 'type-from-yaml'
scope: 'scope-from-yaml'
17 changes: 0 additions & 17 deletions tests/config/test_DeployConfig.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
"snowflake_schema": "some_snowflake_schema",
"change_history_table": "some_history_table",
"query_tag": "some_query_tag",
"oauth_config": {"some": "values"},
}


Expand Down Expand Up @@ -136,22 +135,6 @@ def test_check_for_deploy_args_oauth_with_file_happy_path(_):
assert config.snowflake_oauth_token == "my-oauth-token-from-a-file"


@mock.patch("schemachange.config.DeployConfig.get_oauth_token")
def test_check_for_deploy_args_oauth_with_request_happy_path(mock_get_oauth_token):
oauth_token = "my-oauth-token-from-a-request"
mock_get_oauth_token.return_value = oauth_token
oauth_config = {"my_oauth_config": "values"}
config = DeployConfig.factory(
**minimal_deploy_config_kwargs,
snowflake_authenticator="oauth",
oauth_config=oauth_config,
config_file_path=Path("."),
)
config.check_for_deploy_args()
assert config.snowflake_oauth_token == oauth_token
mock_get_oauth_token.call_args.args[0] == oauth_config


def test_check_for_deploy_args_externalbrowser_happy_path():
config = DeployConfig.factory(
**minimal_deploy_config_kwargs,
Expand Down
Loading

0 comments on commit 1beb3a5

Please sign in to comment.