Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to latest django-storages and add support for Azure Blob Storage #3794

Merged
merged 3 commits into from
May 4, 2022

Conversation

bufke
Copy link
Contributor

@bufke bufke commented May 3, 2022

Description

Upgrade django-storages package. Adds support for Azure Blob Storage, outside of django-private-storage

Checklist

  • Upgrade to latest django-storages and port tellable storage workaround
  • Add support for Azure blob storage
  • Test in Azure
  • Test in S3

This PR does not address django-private-storage continuing to use only S3. Therefor this PR only partially support Azure.

Related to kobotoolbox/kobocat#816

@bufke bufke requested a review from noliveleger May 4, 2022 16:33
@bufke bufke marked this pull request as ready for review May 4, 2022 16:33
@@ -591,7 +591,7 @@ def absolute_mp3_path(self):
content = self.get_mp3_content()
kobocat_storage.save(self.mp3_storage_path, ContentFile(content))

if not isinstance(kobocat_storage, KobocatS3Boto3Storage):
if isinstance(kobocat_storage, KobocatFileSystemStorage):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reversed this to make anything not local storage, use the S3 like storage. Which maybe works for azure too. I really hate this but it's the only quick way I could think to make it work. It should really not be so S3 specific and just happen to maybe work in other places.

DEFAULT_FILE_STORAGE = os.environ.get('KPI_DEFAULT_FILE_STORAGE')
if DEFAULT_FILE_STORAGE == 'storages.backends.s3boto3.S3Boto3Storage':
# Force usage of custom S3 tellable Storage
DEFAULT_FILE_STORAGE = 'kobo.apps.storage_backends.s3boto3.S3Boto3Storage'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'd rather no one know we do this horrible hack 😄 and we may some day be able to remove it. So we don't want under users changing it.

bucket = django_settings.KOBOCAT_AWS_STORAGE_BUCKET_NAME
super().__init__(acl=None, bucket=bucket, **settings)
super().__init__(*args, **kwargs)
Copy link
Contributor

@noliveleger noliveleger May 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 bucket should be enforced to django_settings.KOBOCAT_AWS_STORAGE_BUCKET_NAME, right?
Am I missing something or it's not passed to parent anymore?

Should not be something like this instead?:

    def __init__(self, *args, **kwargs):
        kwargs['bucket_name'] = django_settings.KOBOCAT_AWS_STORAGE_BUCKET_NAME
        super().__init__(*args, **kwargs)

BTW, it should be bucket_name now, bucket is deprecated. I did not notice it when I created the class. I have changed it in the Django 3.2 branch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, I tested by making an alternative bucket name for kc


class KobocatAzureStorage(AzureStorage):
def __init__(self, *args, **kwargs):
azure_container = django_settings.KOBOCAT_AZURE_CONTAINER
Copy link
Contributor

@noliveleger noliveleger May 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Same thoughts as above? How does KobocatAzureStorage know when to write since azure_container is not passed to the parent constructor?

Should not be something like this instead?:

    def __init__(self, *args, **kwargs):
        kwargs['azure_container'] = django_settings. KOBOCAT_AZURE_CONTAINER
        super().__init__(*args, **kwargs)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed entirely as discussed. We won't support an alternative kc azure container.

Remove support for having seperate kobocat azure container for kobocat
@bufke bufke requested a review from noliveleger May 4, 2022 20:30
@noliveleger noliveleger merged commit 5e4806f into beta May 4, 2022
@noliveleger noliveleger deleted the azure-blob branch May 4, 2022 20:50
@jnm
Copy link
Member

jnm commented May 5, 2022

@bufke bufke mentioned this pull request May 5, 2022
@jnm jnm changed the title Upgraded to latest django-storages and inserted azure blob settings Upgrade to latest django-storages and inserted azure blob settings Jul 13, 2022
@jnm jnm changed the title Upgrade to latest django-storages and inserted azure blob settings Upgrade to latest django-storages and add support for Azure Blob Storage Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants