Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logging #43

Merged
merged 5 commits into from
Feb 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## v3.0.1 - 2021-02-12

### Changed
- The `wrangler.py` module now has logging. To observe the arguments and sql queries being run, turn on logging at DEBUG level using e.g.

```
import logging
logging.basicConfig()
logging.getLogger("pydbtools").setLevel(logging.DEBUG)
```
## v3.0.0 - 2021-01-26

### Changed
Expand Down
10 changes: 8 additions & 2 deletions pydbtools/wrangler.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
import os
import sqlparse
import warnings
import logging
import pprint

import inspect
import functools
Expand All @@ -17,6 +19,9 @@
replace_temp_database_name_reference,
)


logger = logging.getLogger(__name__)

# Wrapper used to set parameters in the athena wrangler functions
# before they are called
def init_athena_params(func=None, *, allow_boto3_session=False):
Expand All @@ -26,7 +31,7 @@ def init_athena_params(func=None, *, allow_boto3_session=False):

Args:
func (Callable): An function from wr.athena that requires
boto3_session. If the func has an s3_output this is also
boto3_session. If the func has an s3_output this is also
standardised.

Returns:
Expand Down Expand Up @@ -109,7 +114,8 @@ def wrapper(*args, **kwargs):
argmap["database"] = temp_db_name
else:
argmap["database"] = None

logger.debug(f"Modifying function {func.__name__}")
logger.debug(pprint.pformat(dict(argmap)))
return func(**argmap)

return wrapper
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[tool]
[tool.poetry]
name = "pydbtools"
version = "3.0.0"
version = "3.0.1"
description = "A python package to query data via amazon athena and bring it into a pandas df"
license = "MIT"
authors = ["Karik Isichei <[email protected]>"]
Expand Down