Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python3 and Django2.2 upgrade] part 6 - Settings files refactoring #683

Merged
merged 34 commits into from
Mar 12, 2021
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
7deb24b
Moved common settings to "base.py"
noliveleger Aug 18, 2020
020183a
Removed scheduled task for tmp issue 242
noliveleger Aug 18, 2020
9a6e7d2
Removed several settings from `base.py`:
noliveleger Aug 18, 2020
1a168e7
Removed raven package. Use sentry-sdk instead
noliveleger Aug 18, 2020
542a718
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 18, 2020
682dc70
Removed django-nose and raven pip packages. Added sentry-sdk
noliveleger Aug 19, 2020
0cdddf8
Reorganized settings
noliveleger Aug 19, 2020
5c3aaf1
Removed not used ExceptionLoggingMiddleware
noliveleger Aug 19, 2020
5a74dff
Removed installation of raven package on dev mode
noliveleger Aug 19, 2020
cde9f0b
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 19, 2020
ef851c0
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 19, 2020
2dad434
Removed South migrations
noliveleger Aug 20, 2020
587e525
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 20, 2020
0b6958c
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 20, 2020
ab65d65
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 20, 2020
bb69e77
fixed conflicts after merge
noliveleger Aug 20, 2020
7d98845
Changed SECRET_KEY in pytest.ini to random string to be replace withi…
noliveleger Aug 20, 2020
b19a9b1
Removed duplicated tests with bad imports
noliveleger Aug 20, 2020
a17b9d8
Get rid of "KPI_URL" in favor of "KOBOFORM_URL"
noliveleger Aug 20, 2020
7bdc8d1
Redirect anonymous users to KPI when they try to download XLSForm
noliveleger Aug 20, 2020
e87b589
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Aug 25, 2020
8f1ad36
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Sep 22, 2020
95cccea
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Sep 22, 2020
234dae6
Force Django-Guardian AnonymousUser to use -1 as PK
noliveleger Oct 7, 2020
b1b1f17
Allow UWSGI_USER to write into "log" and "backup" folders
noliveleger Oct 7, 2020
d31485b
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Oct 8, 2020
c8df68e
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Dec 17, 2020
37cd29e
Fixed: Django-nose/nose pip dependencies are missing
noliveleger Dec 18, 2020
5837b53
Fixed: ODK validator cannot parse Java version with revision number (…
noliveleger Dec 18, 2020
9bca32f
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Dec 18, 2020
6ec6e21
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Jan 29, 2021
6a45ad3
Merge branch 'kobokitten-py3-new-docker-image' into kobokitten-py3-dj…
noliveleger Feb 9, 2021
81e1d52
Merge branch 'kobocat2.0-step5-new-docker-image' into kobocat2.0-step…
noliveleger Mar 4, 2021
ac1ce8c
Merge branch 'kobocat2.0-step5-new-docker-image' into kobocat2.0-step…
noliveleger Mar 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 135 additions & 33 deletions onadata/settings/base.py
Original file line number Diff line number Diff line change
@@ -1,28 +1,19 @@
# coding: utf-8
# vim: set fileencoding=utf-8
# this system uses structured settings as defined in
# http://www.slideshare.net/jacobian/the-best-and-worst-of-django
#
# this is the base settings.py -- which contains settings common to all
# implementations of ona: edit it at last resort
#
# local customizations should be done in several files each of which in turn
# imports this one.
# The local files should be used as the value for your DJANGO_SETTINGS_FILE
# environment variable as needed.
import logging
import multiprocessing
import os
import subprocess # nopep8, used by included files
import sys # nopep8, used by included files

import dj_database_url
from django.conf.global_settings import PASSWORD_HASHERS
from django.core.exceptions import SuspiciousOperation
from django.utils.six import string_types
from django.utils.six.moves.urllib.parse import quote_plus
from pymongo import MongoClient
from pyxform.xform2json import logger

from onadata.libs.utils.redis_helper import RedisHelper
from pyxform.xform2json import logger

BASE_DIR = os.path.dirname(os.path.dirname(__file__))
ONADATA_DIR = BASE_DIR
@@ -38,7 +29,7 @@
MANAGERS = ADMINS


DEFAULT_FROM_EMAIL = 'noreply@ona.io'
DEFAULT_FROM_EMAIL = 'noreply@kobotoolbox.org'
DEFAULT_SESSION_EXPIRY_TIME = 21600 # 6 hours

# Local time zone for this installation. Choices can be found here:
@@ -69,7 +60,7 @@
# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash.
# Examples: "http://media.lawrence.com/media/", "http://example.com/media/"
MEDIA_URL = 'http://localhost:8000/media/'
MEDIA_URL = f"/{os.environ.get('KOBOCAT_MEDIA_URL', 'media').strip('/')}/"

# Absolute path to the directory static files should be collected to.
# Don't put anything in this directory yourself; store your static files
@@ -89,25 +80,21 @@

ENKETO_URL = ENKETO_URL.rstrip('/')
ENKETO_API_TOKEN = os.environ.get('ENKETO_API_TOKEN', 'enketorules')
ENKETO_VERSION = os.environ.get('ENKETO_VERSION', 'Legacy').lower()
assert ENKETO_VERSION in ['legacy', 'express']
ENKETO_VERSION = 'express'

# Constants.
ENKETO_API_ENDPOINT_ONLINE_SURVEYS = '/survey'
ENKETO_API_ENDPOINT_OFFLINE_SURVEYS = '/survey/offline'
ENKETO_API_ENDPOINT_INSTANCE = '/instance'
ENKETO_API_ENDPOINT_INSTANCE_IFRAME = '/instance/iframe'

# Computed settings.
if ENKETO_VERSION == 'express':
ENKETO_API_ROOT = '/api/v2'
ENKETO_OFFLINE_SURVEYS = os.environ.get('ENKETO_OFFLINE_SURVEYS', 'True').lower() == 'true'
ENKETO_API_ENDPOINT_PREVIEW = '/preview'
ENKETO_API_ENDPOINT_SURVEYS = ENKETO_API_ENDPOINT_OFFLINE_SURVEYS if ENKETO_OFFLINE_SURVEYS \
else ENKETO_API_ENDPOINT_ONLINE_SURVEYS
else:
ENKETO_API_ROOT= '/api_v1'
ENKETO_API_ENDPOINT_PREVIEW = '/webform/preview'
ENKETO_OFFLINE_SURVEYS = False
ENKETO_API_ENDPOINT_SURVEYS = ENKETO_API_ENDPOINT_ONLINE_SURVEYS
ENKETO_API_ROOT = '/api/v2'
ENKETO_OFFLINE_SURVEYS = os.environ.get('ENKETO_OFFLINE_SURVEYS', 'True').lower() == 'true'
ENKETO_API_ENDPOINT_PREVIEW = '/preview'
ENKETO_API_ENDPOINT_SURVEYS = ENKETO_API_ENDPOINT_OFFLINE_SURVEYS if ENKETO_OFFLINE_SURVEYS \
else ENKETO_API_ENDPOINT_ONLINE_SURVEYS

ENKETO_API_SURVEY_PATH = ENKETO_API_ROOT + ENKETO_API_ENDPOINT_SURVEYS
ENKETO_API_INSTANCE_PATH = ENKETO_API_ROOT + ENKETO_API_ENDPOINT_INSTANCE
ENKETO_PREVIEW_URL = ENKETO_URL + ENKETO_API_ENDPOINT_PREVIEW
@@ -153,6 +140,7 @@
]

MIDDLEWARE = [
'onadata.koboform.redirect_middleware.ConditionalRedirects',
'reversion.middleware.RevisionMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
@@ -186,6 +174,7 @@
],
'OPTIONS': {
'context_processors': [
'onadata.koboform.context_processors.koboform_integration',
'django.contrib.auth.context_processors.auth',
'django.template.context_processors.debug',
'django.template.context_processors.i18n',
@@ -439,9 +428,12 @@ def skip_suspicious_operations(record):
REST_SERVICE_MAX_RETRIES = 3

# celery
CELERY_BROKER_URL = 'redis://localhost:6389/2'
CELERY_RESULT_BACKEND = 'redis://localhost:6389/2' # telling Celery to report results to Redis
CELERY_TASK_ALWAYS_EAGER = False
CELERY_BROKER_URL = os.environ.get(
'KOBOCAT_BROKER_URL', 'redis://localhost:6389/2')

CELERY_RESULT_BACKEND = CELERY_BROKER_URL

CELERY_TASK_ALWAYS_EAGER = os.environ.get('SKIP_CELERY', 'False') == 'True'

# Celery defaults to having as many workers as there are cores. To avoid
# excessive resource consumption, don't spawn more than 6 workers by default
@@ -467,6 +459,16 @@ def skip_suspicious_operations(record):
"visibility_timeout": 120 * (10 ** REST_SERVICE_MAX_RETRIES) # Longest ETA for RestService
}

CELERY_BEAT_SCHEDULE = {
# Periodically mark exports stuck in the "pending" state as "failed"
# See https://github.com/kobotoolbox/kobocat/issues/315
'log-stuck-exports-and-mark-failed': {
'task': 'onadata.apps.viewer.tasks.log_stuck_exports_and_mark_failed',
'schedule': timedelta(hours=6),
'options': {'queue': 'kobocat_queue'}
},
}

CELERY_TASK_DEFAULT_QUEUE = "kobocat_queue"

# duration to keep zip exports before deletion (in seconds)
@@ -478,9 +480,7 @@ def skip_suspicious_operations(record):
TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'
NOSE_ARGS = ['--with-fixture-bundling']

# fake endpoints for testing
TEST_HTTP_HOST = 'testserver.com'
TEST_USERNAME = 'bob'


# re-captcha in registrations
REGISTRATION_REQUIRE_CAPTCHA = False
@@ -591,6 +591,108 @@ def skip_suspicious_operations(record):
# The maximum size (in bytes) that an upload will be before it gets streamed to the file system
FILE_UPLOAD_MAX_MEMORY_SIZE = 10485760

# MongoDB
MONGO_DATABASE = {
'HOST': os.environ.get('KOBOCAT_MONGO_HOST', 'mongo'),
'PORT': int(os.environ.get('KOBOCAT_MONGO_PORT', 27017)),
'NAME': os.environ.get('KOBOCAT_MONGO_NAME', 'formhub'),
'USER': os.environ.get('KOBOCAT_MONGO_USER', ''),
'PASSWORD': os.environ.get('KOBOCAT_MONGO_PASS', '')
}

if MONGO_DATABASE.get('USER') and MONGO_DATABASE.get('PASSWORD'):
MONGO_CONNECTION_URL = "mongodb://{user}:{password}@{host}:{port}/{db_name}".\
format(
user=MONGO_DATABASE['USER'],
password=quote_plus(MONGO_DATABASE['PASSWORD']),
host=MONGO_DATABASE['HOST'],
port=MONGO_DATABASE['PORT'],
db_name=MONGO_DATABASE['NAME']
)
else:
MONGO_CONNECTION_URL = "mongodb://%(HOST)s:%(PORT)s/%(NAME)s" % MONGO_DATABASE

# PyMongo 3 does acknowledged writes by default
# https://emptysqua.re/blog/pymongos-new-default-safe-writes/
MONGO_CONNECTION = MongoClient(
MONGO_CONNECTION_URL, j=True, tz_aware=True)

MONGO_DB = MONGO_CONNECTION[MONGO_DATABASE['NAME']]

LOCALE_PATHS = [os.path.join(PROJECT_ROOT, 'locale'), ]

DEBUG = os.environ.get('DJANGO_DEBUG', 'True') == 'True'
TEMPLATE_DEBUG = os.environ.get('TEMPLATE_DEBUG', 'True') == 'True'

# Database (i.e. PostgreSQL)
DATABASES = {
'default': dj_database_url.config(default="sqlite:///%s/db.sqlite3" % PROJECT_ROOT)
}
# Replacement for TransactionMiddleware
DATABASES['default']['ATOMIC_REQUESTS'] = True

# Django `SECRET_KEY`
try:
SECRET_KEY = os.environ['DJANGO_SECRET_KEY']
except KeyError:
raise Exception('DJANGO_SECRET_KEY must be set in the environment.')

ALLOWED_HOSTS = os.environ.get('DJANGO_ALLOWED_HOSTS', '*').split(' ')

TESTING_MODE = False

# Domain must not exclude KPI when sharing sessions
if os.environ.get('SESSION_COOKIE_DOMAIN'):
SESSION_COOKIE_DOMAIN = os.environ['SESSION_COOKIE_DOMAIN']
SESSION_COOKIE_NAME = 'kobonaut'

SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'

KOBOFORM_SERVER = os.environ.get("KOBOFORM_SERVER", "localhost")
KOBOFORM_SERVER_PORT = os.environ.get("KOBOFORM_SERVER_PORT", "8000")
KOBOFORM_SERVER_PROTOCOL = os.environ.get("KOBOFORM_SERVER_PROTOCOL", "http")
KOBOFORM_LOGIN_AUTOREDIRECT = True
KOBOFORM_URL = os.environ.get("KOBOFORM_URL", "http://kf.kobo.local")
KOBOCAT_URL = os.environ.get("KOBOCAT_URL", "http://kc.kobo.local")

# BEGIN external service integration codes
AWS_ACCESS_KEY_ID = os.environ.get('KOBOCAT_AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('KOBOCAT_AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('KOBOCAT_AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'private'
AWS_S3_FILE_BUFFER_SIZE = 50 * 1024 * 1024

# TODO pass these variables from `kobo-docker` envfiles
AWS_QUERYSTRING_EXPIRE = os.environ.get("KOBOCAT_AWS_QUERYSTRING_EXPIRE", 3600)
AWS_S3_USE_SSL = os.environ.get("KOBOCAT_AWS_S3_USE_SSL", True)
AWS_S3_HOST = os.environ.get("KOBOCAT_AWS_S3_HOST", "s3.amazonaws.com")

GOOGLE_ANALYTICS_PROPERTY_ID = os.environ.get("GOOGLE_ANALYTICS_TOKEN", False)
GOOGLE_ANALYTICS_DOMAIN = "auto"
# END external service integration codes

# If not properly overridden, leave uninitialized so Django can set the default.
# (see https://docs.djangoproject.com/en/1.8/ref/settings/#default-file-storage)
if os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE'):
DEFAULT_FILE_STORAGE = os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE')

EMAIL_BACKEND = os.environ.get('EMAIL_BACKEND',
'django.core.mail.backends.filebased.EmailBackend')

if EMAIL_BACKEND == 'django.core.mail.backends.filebased.EmailBackend':
EMAIL_FILE_PATH = os.environ.get(
'EMAIL_FILE_PATH', os.path.join(PROJECT_ROOT, 'emails'))
if not os.path.isdir(EMAIL_FILE_PATH):
os.mkdir(EMAIL_FILE_PATH)

# Default value for the `UserProfile.require_auth` attribute
REQUIRE_AUTHENTICATION_TO_SEE_FORMS_AND_SUBMIT_DATA_DEFAULT = os.environ.get(
'REQUIRE_AUTHENTICATION_TO_SEE_FORMS_AND_SUBMIT_DATA_DEFAULT',
'False') == 'True'

POSTGIS_VERSION = (2, 5, 4)

# Monkey Patch PyXForm. @ToDo remove after upgrading to v1.1.0
logger.removeHandler(logging.NullHandler)
logger.addHandler(logging.NullHandler())

2 changes: 2 additions & 0 deletions onadata/settings/dev.py
Original file line number Diff line number Diff line change
@@ -11,3 +11,5 @@
'handlers': ['console'],
'level': 'DEBUG'
}

MIDDLEWARE.append('onadata.libs.utils.middleware.ExceptionLoggingMiddleware')
167 changes: 3 additions & 164 deletions onadata/settings/prod.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
# coding: utf-8
from datetime import timedelta
import logging
import os

import dj_database_url

from celery.signals import after_setup_logger
from django.utils.six.moves.urllib.parse import quote_plus

from onadata.settings.base import *
from .base import *


def celery_logger_setup_handler(logger, **kwargs):
@@ -23,160 +20,14 @@ def celery_logger_setup_handler(logger, **kwargs):
logger.addHandler(my_handler)


LOCALE_PATHS = [os.path.join(PROJECT_ROOT, 'locale'), ]

DEBUG = os.environ.get('DJANGO_DEBUG', 'True') == 'True'
TEMPLATE_DEBUG = os.environ.get('TEMPLATE_DEBUG', 'True') == 'True'
TEMPLATE_STRING_IF_INVALID = ''


DATABASES = {
'default': dj_database_url.config(default="sqlite:///%s/db.sqlite3" % PROJECT_ROOT)
}
# Replacement for TransactionMiddleware
DATABASES['default']['ATOMIC_REQUESTS'] = True

MONGO_DATABASE = {
'HOST': os.environ.get('KOBOCAT_MONGO_HOST', 'mongo'),
'PORT': int(os.environ.get('KOBOCAT_MONGO_PORT', 27017)),
'NAME': os.environ.get('KOBOCAT_MONGO_NAME', 'formhub'),
'USER': os.environ.get('KOBOCAT_MONGO_USER', ''),
'PASSWORD': os.environ.get('KOBOCAT_MONGO_PASS', '')
}

CELERY_BROKER_URL = os.environ.get(
'KOBOCAT_BROKER_URL', 'amqp://guest:guest@rabbit:5672/')

CELERY_RESULT_BACKEND = CELERY_BROKER_URL

try:
SECRET_KEY = os.environ['DJANGO_SECRET_KEY']
except KeyError:
raise Exception('DJANGO_SECRET_KEY must be set in the environment.')

ALLOWED_HOSTS = os.environ.get('DJANGO_ALLOWED_HOSTS', '*').split(' ')

TESTING_MODE = False
# This trick works only when we run tests from the command line.
if len(sys.argv) >= 2 and (sys.argv[1] == "test"):
raise Exception(
"Testing destroys data and must NOT be run in a production "
"environment. Please use a different settings file if you want to "
"run tests."
)
TESTING_MODE = True
else:
TESTING_MODE = False

MEDIA_URL = f"/{os.environ.get('KOBOCAT_MEDIA_URL', 'media').strip('/')}/"
STATIC_URL = '/static/'
LOGIN_URL = '/accounts/login/'
LOGIN_REDIRECT_URL = '/login_redirect/'

if os.environ.get('KOBOCAT_ROOT_URI_PREFIX'):
KOBOCAT_ROOT_URI_PREFIX = '/' + os.environ['KOBOCAT_ROOT_URI_PREFIX'].strip('/') + '/'
MEDIA_URL = KOBOCAT_ROOT_URI_PREFIX + MEDIA_URL.lstrip('/')
STATIC_URL = KOBOCAT_ROOT_URI_PREFIX + STATIC_URL.lstrip('/')
LOGIN_URL = KOBOCAT_ROOT_URI_PREFIX + LOGIN_URL.lstrip('/')
LOGIN_REDIRECT_URL = KOBOCAT_ROOT_URI_PREFIX + LOGIN_REDIRECT_URL.lstrip('/')

if TESTING_MODE:
MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'test_media/')
subprocess.call(["rm", "-r", MEDIA_ROOT])
MONGO_DATABASE['NAME'] = "formhub_test"
CELERY_TASK_ALWAYS_EAGER = True
BROKER_BACKEND = 'memory'
ENKETO_API_TOKEN = 'abc'
#TEST_RUNNER = 'djcelery.contrib.test_runner.CeleryTestSuiteRunner'
else:
MEDIA_ROOT = os.path.join(PROJECT_ROOT, MEDIA_URL.lstrip('/'))

if PRINT_EXCEPTION and DEBUG:
MIDDLEWARE.append('onadata.libs.utils.middleware.ExceptionLoggingMiddleware')

# Clear out the test database
if TESTING_MODE:
MONGO_DB.instances.drop()

KOBOFORM_SERVER = os.environ.get("KOBOFORM_SERVER", "localhost")
KOBOFORM_SERVER_PORT = os.environ.get("KOBOFORM_SERVER_PORT", "8000")
KOBOFORM_SERVER_PROTOCOL = os.environ.get("KOBOFORM_SERVER_PROTOCOL", "http")
KOBOFORM_LOGIN_AUTOREDIRECT = True
KOBOFORM_URL = os.environ.get("KOBOFORM_URL", "http://kf.kobo.local")
KOBOCAT_URL = os.environ.get("KOBOCAT_URL", "http://kc.kobo.local")


TEMPLATES[0]['OPTIONS']['context_processors'].insert(
0, 'onadata.koboform.context_processors.koboform_integration')

MIDDLEWARE.insert(0, 'onadata.koboform.redirect_middleware.ConditionalRedirects')

# Domain must not exclude KPI when sharing sessions
if os.environ.get('SESSION_COOKIE_DOMAIN'):
SESSION_COOKIE_DOMAIN = os.environ['SESSION_COOKIE_DOMAIN']
SESSION_COOKIE_NAME = 'kobonaut'

SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'

# for debugging
# print "KOBOFORM_URL=%s" % KOBOFORM_URL
# print "SECRET_KEY=%s" % SECRET_KEY
# print "CSRF_COOKIE_DOMAIN=%s " % CSRF_COOKIE_DOMAIN

# MongoDB - moved here from base.py
if MONGO_DATABASE.get('USER') and MONGO_DATABASE.get('PASSWORD'):
MONGO_CONNECTION_URL = "mongodb://{user}:{password}@{host}:{port}/{db_name}".\
format(
user=MONGO_DATABASE['USER'],
password=quote_plus(MONGO_DATABASE['PASSWORD']),
host=MONGO_DATABASE['HOST'],
port=MONGO_DATABASE['PORT'],
db_name=MONGO_DATABASE['NAME']
)
else:
MONGO_CONNECTION_URL = "mongodb://%(HOST)s:%(PORT)s/%(NAME)s" % MONGO_DATABASE

# PyMongo 3 does acknowledged writes by default
# https://emptysqua.re/blog/pymongos-new-default-safe-writes/
MONGO_CONNECTION = MongoClient(
MONGO_CONNECTION_URL, j=True, tz_aware=True)

MONGO_DB = MONGO_CONNECTION[MONGO_DATABASE['NAME']]

# BEGIN external service integration codes
AWS_ACCESS_KEY_ID = os.environ.get('KOBOCAT_AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('KOBOCAT_AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('KOBOCAT_AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'private'
AWS_S3_FILE_BUFFER_SIZE = 50 * 1024 * 1024
# TODO pass these variables from `kobo-docker` envfiles
AWS_QUERYSTRING_EXPIRE = os.environ.get("KOBOCAT_AWS_QUERYSTRING_EXPIRE", 3600)
AWS_S3_USE_SSL = os.environ.get("KOBOCAT_AWS_S3_USE_SSL", True)
AWS_S3_HOST = os.environ.get("KOBOCAT_AWS_S3_HOST", "s3.amazonaws.com")


GOOGLE_ANALYTICS_PROPERTY_ID = os.environ.get("GOOGLE_ANALYTICS_TOKEN", False)
GOOGLE_ANALYTICS_DOMAIN = "auto"
# END external service integration codes

# If not properly overridden, leave uninitialized so Django can set the default.
# (see https://docs.djangoproject.com/en/1.8/ref/settings/#default-file-storage)
if os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE'):
DEFAULT_FILE_STORAGE = os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE')

EMAIL_BACKEND = os.environ.get('EMAIL_BACKEND',
'django.core.mail.backends.filebased.EmailBackend')

if EMAIL_BACKEND == 'django.core.mail.backends.filebased.EmailBackend':
EMAIL_FILE_PATH = os.environ.get(
'EMAIL_FILE_PATH', os.path.join(PROJECT_ROOT, 'emails'))
if not os.path.isdir(EMAIL_FILE_PATH):
os.mkdir(EMAIL_FILE_PATH)

# Default value for the `UserProfile.require_auth` attribute
REQUIRE_AUTHENTICATION_TO_SEE_FORMS_AND_SUBMIT_DATA_DEFAULT = os.environ.get(
'REQUIRE_AUTHENTICATION_TO_SEE_FORMS_AND_SUBMIT_DATA_DEFAULT',
'False') == 'True'
MEDIA_ROOT = os.path.join(PROJECT_ROOT, MEDIA_URL.lstrip('/'))

# Optional Sentry configuration: if desired, be sure to install Raven and set
# RAVEN_DSN in the environment
@@ -259,17 +110,6 @@ def celery_logger_setup_handler(logger, **kwargs):
CELERY_WORKER_HIJACK_ROOT_LOGGER = False
after_setup_logger.connect(celery_logger_setup_handler)

POSTGIS_VERSION = (2, 5, 0)

CELERY_BEAT_SCHEDULE = {
# Periodically mark exports stuck in the "pending" state as "failed"
# See https://github.com/kobotoolbox/kobocat/issues/315
'log-stuck-exports-and-mark-failed': {
'task': 'onadata.apps.viewer.tasks.log_stuck_exports_and_mark_failed',
'schedule': timedelta(hours=6),
'options': {'queue': 'kobocat_queue'}
},
}

# ## ISSUE 242 TEMPORARY FIX ###
# See https://github.com/kobotoolbox/kobocat/issues/242
@@ -291,4 +131,3 @@ def celery_logger_setup_handler(logger, **kwargs):

SESSION_ENGINE = "redis_sessions.session"
SESSION_REDIS = RedisHelper.config(default="redis://redis_cache:6380/2")

101 changes: 5 additions & 96 deletions onadata/settings/testing.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
# coding: utf-8
import dj_database_url

from mongomock import MongoClient as MockMongoClient

from onadata.settings.base import *

DEBUG = os.environ.get('DJANGO_DEBUG', 'True') == 'True'
TEMPLATE_DEBUG = os.environ.get('TEMPLATE_DEBUG', 'True') == 'True'
TEMPLATE_STRING_IF_INVALID = ''

from .base import *

DATABASES = {
'default': dj_database_url.config(
@@ -27,106 +21,21 @@
MONGO_CONNECTION_URL, j=True, tz_aware=True)
MONGO_DB = MONGO_CONNECTION['formhub_test']


CELERY_BROKER_URL = os.environ.get(
'KOBOCAT_BROKER_URL', 'amqp://guest:guest@rabbit:5672/')

try:
SECRET_KEY = os.environ['DJANGO_SECRET_KEY']
except KeyError:
raise Exception('DJANGO_SECRET_KEY must be set in the environment.')

ALLOWED_HOSTS = os.environ.get('DJANGO_ALLOWED_HOSTS', '*').split(' ')

TESTING_MODE = True

MEDIA_URL = '/' + os.environ.get('KOBOCAT_MEDIA_URL', 'media').strip('/') + '/'
STATIC_URL = '/static/'
LOGIN_URL = '/accounts/login/'
LOGIN_REDIRECT_URL = '/login_redirect/'

if os.environ.get('KOBOCAT_ROOT_URI_PREFIX'):
KOBOCAT_ROOT_URI_PREFIX = '/' + os.environ['KOBOCAT_ROOT_URI_PREFIX'].strip('/') + '/'
MEDIA_URL = KOBOCAT_ROOT_URI_PREFIX + MEDIA_URL.lstrip('/')
STATIC_URL = KOBOCAT_ROOT_URI_PREFIX + STATIC_URL.lstrip('/')
LOGIN_URL = KOBOCAT_ROOT_URI_PREFIX + LOGIN_URL.lstrip('/')
LOGIN_REDIRECT_URL = KOBOCAT_ROOT_URI_PREFIX + LOGIN_REDIRECT_URL.lstrip('/')

MEDIA_ROOT = '/tmp/test_media/'
CELERY_TASK_ALWAYS_EAGER = True
BROKER_BACKEND = 'memory'
ENKETO_API_TOKEN = 'abc'

if PRINT_EXCEPTION and DEBUG:
MIDDLEWARE.append('onadata.libs.utils.middleware.ExceptionLoggingMiddleware')

KOBOFORM_SERVER = os.environ.get("KOBOFORM_SERVER", "localhost")
KOBOFORM_SERVER_PORT = os.environ.get("KOBOFORM_SERVER_PORT", "8000")
KOBOFORM_SERVER_PROTOCOL = os.environ.get("KOBOFORM_SERVER_PROTOCOL", "http")
# KOBOFORM_LOGIN_AUTOREDIRECT=True
KOBOFORM_URL = os.environ.get("KOBOFORM_URL", "http://localhost:8000")

TEMPLATES[0]['OPTIONS']['context_processors'].append('onadata.koboform.context_processors.koboform_integration')

# Domain must not exclude KPI when sharing sessions
if os.environ.get('SESSION_COOKIE_DOMAIN'):
SESSION_COOKIE_DOMAIN = os.environ['SESSION_COOKIE_DOMAIN']
SESSION_COOKIE_NAME = 'kobonaut'

SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'

# for debugging
# print "KOBOFORM_URL=%s" % KOBOFORM_URL
# print "SECRET_KEY=%s" % SECRET_KEY
# print "CSRF_COOKIE_DOMAIN=%s " % CSRF_COOKIE_DOMAIN

# BEGIN external service integration codes
AWS_ACCESS_KEY_ID = os.environ.get('KOBOCAT_AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('KOBOCAT_AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('KOBOCAT_AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'private'

GOOGLE_ANALYTICS_PROPERTY_ID = os.environ.get("GOOGLE_ANALYTICS_TOKEN", False)
GOOGLE_ANALYTICS_DOMAIN = "auto"
# END external service integration codes

# If not properly overridden, leave uninitialized so Django can set the default.
# (see https://docs.djangoproject.com/en/1.8/ref/settings/#default-file-storage)
if os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE'):
DEFAULT_FILE_STORAGE = os.environ.get('KOBOCAT_DEFAULT_FILE_STORAGE')

EMAIL_BACKEND = os.environ.get('EMAIL_BACKEND',
'django.core.mail.backends.filebased.EmailBackend')

if EMAIL_BACKEND == 'django.core.mail.backends.filebased.EmailBackend':
EMAIL_FILE_PATH = os.environ.get(
'EMAIL_FILE_PATH', os.path.join(PROJECT_ROOT, 'emails'))
if not os.path.isdir(EMAIL_FILE_PATH):
os.mkdir(EMAIL_FILE_PATH)

# Optional Sentry configuration: if desired, be sure to install Raven and set
# RAVEN_DSN in the environment
if 'RAVEN_DSN' in os.environ:
try:
import raven
except ImportError:
print('Please install Raven to enable Sentry logging.')
else:
INSTALLED_APPS.append('raven.contrib.django.raven_compat')
RAVEN_CONFIG = {
'dsn': os.environ['RAVEN_DSN'],
}
try:
RAVEN_CONFIG['release'] = raven.fetch_git_sha(BASE_DIR)
except raven.exceptions.InvalidGitRepository:
pass

POSTGIS_VERSION = (2, 5, 0)

# DISABLE Django DB logging
LOGGING['loggers']['django.db.backends'] = {
'level': 'WARNING',
'propagate': True
}

GUARDIAN_GET_INIT_ANONYMOUS_USER = 'onadata.apps.main.models.user_profile.get_anonymous_user_instance'

# fake endpoints for testing
TEST_HTTP_HOST = 'testserver.com'
TEST_USERNAME = 'bob'