Skip to content

Commit

Permalink
feat(conf): ability to determine data folder (#91)
Browse files Browse the repository at this point in the history 
* Add mongod_data_folder to the execution context

* Update README

* Adding tests

* fix mongod test

Signed-off-by: Ertugrul Karademir <[email protected]>

---------

Signed-off-by: Ertugrul Karademir <[email protected]>
  • Loading branch information
@ekarademir
ekarademir authored Oct 28, 2023
1 parent d20eba3 commit 8313537
Show file tree
Hide file tree
Showing 4 changed files with 90 additions and 26 deletions.
26 changes: 14 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ A mongo mocking library with an ephemeral MongoDB running in memory.
## What's new?

### v0.4.0-pre

- Tooling enhancements. [[PR #90](https://github.com/kaizendorks/pymongo_inmemory/pull/90)]

### v0.3.1
Expand Down Expand Up @@ -60,18 +61,19 @@ with MongoClient() as client:

## Configuration

| Config param | Description | Optional? | Default |
| ------------------ | ---------------------------------------------------------------------------------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------- |
| `mongo_version` | Which MongoD version to download and use. | Yes | Latest for the OS |
| `mongod_port` | Override port preference. | Yes | Automatically picked between `27017` and `28000` after testing availability |
| `operating_system` | This makes sense for Linux setting, where there are several flavours | Yes | Automatically determined (Generic for Linux)\* |
| `os_version` | If an operating system has several versions use this parameter to select one | Yes | Latest version of the OS will be selected from the list |
| `download_url` | If set, it won't attempt to determine which MongoDB to download. However there won't be a fallback either. | Yes | Automatically determined from given parameters and using [internal URL bank](pymongo_inmemory/downloader/_patterns.py)\*\* |
| `ignore_cache` | Even if there is a downloaded version in the cache, download it again. | Yes | False |
| `use_local_mongod` | If set, it will try to use a local mongod instance instead of downloading one. | Yes | False |
| `download_folder` | Override the default download location. | Yes | pymongo_inmemory/.cache/download |
| `extract_folder` | Override the default extraction location. | Yes | pymongo_inmemory/.cache/extract |
| | | |
| | Config parameter | Description Default |
| ------- | -------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| | `mongo_version` | Which MongoD version to download and use. | Latest for the OS |
| | `mongod_port` | Override port preference. | Automatically picked between `27017` and `28000` after testing availability |
| | `operating_system` | This makes sense for Linux setting, where there are several flavours | Automatically determined (Generic for Linux)\* |
| | `os_version` | If an operating system has several versions use this parameter to select one | Latest version of the OS will be selected from the list |
| | `download_url` | If set, it won't attempt to determine which MongoDB to download. However there won't be a fallback either. | Automatically determined from given parameters and using [internal URL bank](pymongo_inmemory/downloader/_patterns.py)\*\* |
| | `ignore_cache` | Even if there is a downloaded version in the cache, download it again. | False |
| | `use_local_mongod` | If set, it will try to use a local mongod instance instead of downloading one. | False |
| | `download_folder` | Override the default download location. | pymongo_inmemory/.cache/download |
| | `extract_folder` | Override the default extraction location. | pymongo_inmemory/.cache/extract |
| **NEW** | `mongod_data_folder` | Provide a data folder to be used by MongoD. | A `TemporaryDirectory` will be used |
| | | |

- \***_Note 1:_** Generic Linux version offering for MongoDB ends with version **4.0.23**. If the operating system is just `linux` and if selected MongoDB version is higher, it will default to `4.0.23`.
- **\***Note 2:\*\*\* URL bank is filled with URLs collected from [release list](https://www.mongodb.com/download-center/community/releases) and [archived released list](https://www.mongodb.com/download-center/community/releases/archive), so if a version is not in the bank you can use the same list to provide an official download link.
Expand Down
2 changes: 2 additions & 0 deletions pymongo_inmemory/context.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ def __init__(
) -> None:
self.mongo_version = conf("mongo_version", version)
self.mongod_port = conf("mongod_port", None, coerce_with=int)
self.mongod_data_folder = conf("mongod_data_folder", None)

self.operating_system = self._build_operating_system_info(os_name)
self.os_version = conf("os_version", os_ver)
Expand All @@ -129,6 +130,7 @@ def __str__(self):
return (
f"Mongo Version {self.mongo_version}\n"
f"MongoD Port {self.mongod_port}\n"
f"MongoD Data Folder {self.mongod_data_folder}\n"
f"OS Name {self.operating_system}\n"
f"OS Version {self.os_version}\n"
f"Download URL {self.download_url}\n"
Expand Down
51 changes: 37 additions & 14 deletions pymongo_inmemory/mongod.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,10 @@ def __init__(self, pim_context: Context):
self._connection_string = None

self.config = MongodConfig(self._pim_context)
self.data_folder = TemporaryDirectory(prefix="pymongoim")

self._temp_data_folder = TemporaryDirectory(prefix="pymongoim")
self._using_tmp_folder = self._pim_context.mongod_data_folder is None

self._client = pymongo.MongoClient(self.connection_string)

def __enter__(self):
Expand All @@ -98,22 +101,14 @@ def __exit__(self, *args):
self.stop()

def start(self):
while self.is_locked:
logger.warning(
(
"Lock file found, possibly another mock server is running. "
"Changing the data folder."
)
)
self.data_folder = TemporaryDirectory(prefix="pymongoim")

self.log_path = os.path.join(self.data_folder.name, "mongod.log")
self._check_lock()
self.log_path = os.path.join(self.data_folder, "mongod.log")

logger.info("Starting mongod with {cs}...".format(cs=self.connection_string))
boot_command = [
os.path.join(self._bin_folder, "mongod"),
"--dbpath",
self.data_folder.name,
self.data_folder,
"--logpath",
self.log_path,
"--port",
Expand All @@ -137,7 +132,14 @@ def stop(self):
while self._proc.poll() is None:
logger.debug("Waiting for MongoD shutdown.")
time.sleep(1)
self.data_folder.cleanup()
self._clean_up()

@property
def data_folder(self):
if self._using_tmp_folder:
return self._temp_data_folder.name
else:
return self._pim_context.mongod_data_folder

@property
def connection_string(self):
Expand All @@ -155,7 +157,7 @@ def connection_string(self):

@property
def is_locked(self):
return os.path.exists(os.path.join(self.data_folder.name, "mongod.lock"))
return os.path.exists(os.path.join(self.data_folder, "mongod.lock"))

@property
def is_healthy(self):
Expand Down Expand Up @@ -199,6 +201,27 @@ def logs(self):
with open(self.log_path, "r") as logfile:
return logfile.readlines()

def _clean_up(self):
if self._using_tmp_folder:
self._temp_data_folder.cleanup()

def _check_lock(self):
while self.is_locked:
if self._using_tmp_folder:
raise RuntimeError(
(
"There is a lock file in the provided data folder. "
"Make sure that no other MongoDB is running."
)
)
logger.warning(
(
"Lock file found, possibly another mock server is running. "
"Changing the data folder."
)
)
self._temp_data_folder = TemporaryDirectory(prefix="pymongoim")


if __name__ == "__main__":
# This part is used for integrity tests too.
Expand Down
37 changes: 37 additions & 0 deletions tests/unit/test_mongod.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import subprocess

from pymongo_inmemory.mongod import Mongod
import pymongo_inmemory.downloader as downloader
from pymongo_inmemory.context import Context


class Popen:
def __init__(self, cmd):
self.cmd = cmd
self.terminated = False

def terminate(self):
self.terminated = True

def poll(self):
return True


def returns_true():
return True


def download():
return ""


def test_mongod(monkeypatch):
monkeypatch.setattr(subprocess, "Popen", Popen)
monkeypatch.setattr(Mongod, "is_healthy", returns_true)
monkeypatch.setattr(downloader, "download", download)

context = Context()
context.mongod_data_folder = "TEST"

with Mongod(context) as md:
assert md.data_folder == "TEST"

0 comments on commit 8313537

Please sign in to comment.