Skip to content

Commit

Permalink
Split the docker compose into dev and prod, fixed more issues with th…
Browse files Browse the repository at this point in the history
…e deployment and more frontend improvements

Frontend: More cosmetic fixes and improvements - markdown looks better now
  • Loading branch information
MarianoMolina committed Oct 11, 2024
1 parent 6b8d4ca commit 8949c3c
Show file tree
Hide file tree
Showing 13 changed files with 479 additions and 66 deletions.
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The project consists of three main components:

## Setup and Installation

1. Ensure you have Docker installed on your system. On Windows, it comes with docker-compose, but [check if you have it installed](https://stackoverflow.com/questions/72928891/how-can-i-check-if-docker-compose-plugin-is-installed). Otherwise (if in Linux), [install it](https://docs.docker.com/compose/install/linux/)
1. Ensure you have Docker installed on your system. On Windows, it comes with docker-compose, but [check if you have it installed](https://stackoverflow.com/questions/72928891/how-can-i-check-if-docker-compose-plugin-is-installed). Otherwise (if in Linux), [install it](https://docs.docker.com/compose/install/linux/). If for whatever reason the starting script doesn't start Docker (can't find it), all you need to do is have the Docker Daemon running, for which all you need to do is open your Docker app.

2. (Optional) Install LM Studio if you plan to use local models.

Expand All @@ -28,7 +28,7 @@ The project consists of three main components:
git clone https://github.com/MarianoMolina/project_alice.git
```

4. Create a `.env` file in the root directory using the `template.env` file as a reference. Complete the data for any APIs you want to use (e.g., OpenAI API key).
4. Create a `.env` file in the root directory using the `template.env` file as a reference. Complete the data for any APIs you want to use (e.g., OpenAI API key). Even if you don't update anything, if you don't create it / copy it, the build process will fail.

5. Run the appropriate script for your operating system:
- Windows: Run `run.bat`
Expand Down Expand Up @@ -211,7 +211,7 @@ If you've created new tasks, workflows, or initialization modules that you'd lik
- [Added]: Basic route end code editor

2. **More API engines and base tasks** [Done]: BeautifulSoup to scrap websites, vision_models, text_to_image_models, text_to_speech_models, etc. This will enable a new set of tasks to be created. This includes adding more providers, like Google, Mistral, etc.
- [Added]: 21 new API providers, with their corresponding models
- [Added]: 21 new API providers, with their corresponding models, for a total of 160 distinct entities for you to use.
- [Added]: 2 new workflows -> Web Scrape and Research.

3. **File input and type interface** [Done]: Being able to add files of any type to a conversation, which makes a conversion to text of the file (stt, itt, or simply parsing for files that can be converted to a string), allowing for the user and the agents to share any type of data. This, in turn, requires the agents to also be able to produce different types of outputs, which is where the type interface logic comes in, to convert str -> any and back.
Expand Down Expand Up @@ -242,6 +242,11 @@ If you've created new tasks, workflows, or initialization modules that you'd lik
- Module manager -> Select which modules you want to download and/or keep
- Module integrator -> Select a module to create a fresh version in your DB -> What about removal? Would we want to add a variable to entities to keep track of this?

10. **Cost management**: Currently, the completion metadata is stored, but not much is done with it. Goals are:
- Good tracking of costs
- Task cost estimation based on an algorithm and, when it exists, past data to improve the estimation.
- Cost/use tracking by API in a clear UI

## License

This source code is licensed under the BSD-style license found in the
Expand Down
140 changes: 140 additions & 0 deletions docker-compose.development.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
services:
frontend:
build:
context: ./frontend
dockerfile: Dockerfile.dev
ports:
- "${FRONTEND_PORT}:${FRONTEND_PORT_DOCKER}"
depends_on:
backend:
condition: service_healthy
workflow:
condition: service_healthy
volume-init:
condition: service_completed_successfully
volumes:
- ./frontend:/app:delegated
- /app/node_modules
- /app/dist
- ./.env:/app/.env:ro
environment:
NODE_ENV: ${NODE_ENV:-development}
CHOKIDAR_USEPOLLING: "true"
WATCHPACK_POLLING: "true"

backend:
build:
context: ./backend
dockerfile: Dockerfile.dev
ports:
- "${BACKEND_PORT}:${BACKEND_PORT_DOCKER}"
depends_on:
mongo:
condition: service_started
volume-init:
condition: service_completed_successfully
volumes:
- ./backend:/app
- /app/node_modules
- /app/dist
- ./.env:/app/.env
- logs:/app/logs
- shared-uploads:/app/shared-uploads
environment:
NODE_ENV: ${NODE_ENV:-development}
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:${BACKEND_PORT_DOCKER}/api/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 10s

workflow:
build:
context: ./workflow
env_file:
- .env
ports:
- "${WORKFLOW_PORT}:${WORKFLOW_PORT_DOCKER}"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- logs:/app/logs
- shared-uploads:/app/shared-uploads:ro
depends_on:
volume-init:
condition: service_completed_successfully
python_image:
condition: service_completed_successfully
bash_image:
condition: service_completed_successfully
environment:
- PYTHONPATH=/app
- HOST=host.docker.internal
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:${WORKFLOW_PORT_DOCKER}/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 5s

volume-init:
image: busybox
attach: false
volumes:
- ${COMPOSE_PROJECT_DIR}/logs:/logs
- ${COMPOSE_PROJECT_DIR}/shared-uploads:/shared-uploads
command: |
sh -c "
echo 'Starting volume initialization' &&
ls -la / &&
echo 'Creating directories' &&
mkdir -p /logs /shared-uploads &&
echo 'Setting permissions' &&
chmod -R 755 /logs &&
chown -R 1000:1000 /logs &&
chmod -R 755 /shared-uploads &&
chown -R 1000:1000 /shared-uploads &&
echo 'Finished volume initialization' &&
ls -la /logs /shared-uploads
"
mongo:
image: mongo:latest
ports:
- "26017:27017"
volumes:
- mongo_data:/data/db
attach: false

python_image:
build:
context: .
dockerfile: Dockerfile.python
image: mypython:latest
command: ["bash"]

bash_image:
build:
context: .
dockerfile: Dockerfile.bash
image: mybash:latest
command: ["bash"]

volumes:
logs:
driver: local
driver_opts:
type: none
o: bind
device: ${COMPOSE_PROJECT_DIR}/logs
shared-uploads:
driver: local
driver_opts:
type: none
o: bind
device: ${COMPOSE_PROJECT_DIR}/shared-uploads
mongo_data:
133 changes: 133 additions & 0 deletions docker-compose.production.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
services:
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
ports:
- "${FRONTEND_PORT}:${FRONTEND_PORT_DOCKER}"
depends_on:
backend:
condition: service_healthy
workflow:
condition: service_healthy
volume-init:
condition: service_completed_successfully
volumes:
- ./.env:/app/.env:ro
environment:
NODE_ENV: ${NODE_ENV:-production}
CI: "true"

backend:
build:
context: ./backend
dockerfile: Dockerfile
ports:
- "${BACKEND_PORT}:${BACKEND_PORT_DOCKER}"
depends_on:
mongo:
condition: service_started
volume-init:
condition: service_completed_successfully
volumes:
- ./.env:/app/.env
- logs:/app/logs
- shared-uploads:/app/shared-uploads
environment:
NODE_ENV: ${NODE_ENV:-production}
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:${BACKEND_PORT_DOCKER}/api/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 10s

workflow:
build:
context: ./workflow
env_file:
- .env
ports:
- "${WORKFLOW_PORT}:${WORKFLOW_PORT_DOCKER}"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- logs:/app/logs
- shared-uploads:/app/shared-uploads:ro
depends_on:
volume-init:
condition: service_completed_successfully
python_image:
condition: service_completed_successfully
bash_image:
condition: service_completed_successfully
environment:
- PYTHONPATH=/app
- HOST=host.docker.internal
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:${WORKFLOW_PORT_DOCKER}/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 5s

volume-init:
image: busybox
attach: false
volumes:
- ${COMPOSE_PROJECT_DIR}/logs:/logs
- ${COMPOSE_PROJECT_DIR}/shared-uploads:/shared-uploads
command: |
sh -c "
echo 'Starting volume initialization' &&
ls -la / &&
echo 'Creating directories' &&
mkdir -p /logs /shared-uploads &&
echo 'Setting permissions' &&
chmod -R 755 /logs &&
chown -R 1000:1000 /logs &&
chmod -R 755 /shared-uploads &&
chown -R 1000:1000 /shared-uploads &&
echo 'Finished volume initialization' &&
ls -la /logs /shared-uploads
"
mongo:
image: mongo:latest
ports:
- "26017:27017"
volumes:
- mongo_data:/data/db
attach: false

python_image:
build:
context: .
dockerfile: Dockerfile.python
image: mypython:latest
command: ["bash"]

bash_image:
build:
context: .
dockerfile: Dockerfile.bash
image: mybash:latest
command: ["bash"]

volumes:
logs:
driver: local
driver_opts:
type: none
o: bind
device: ${COMPOSE_PROJECT_DIR}/logs
shared-uploads:
driver: local
driver_opts:
type: none
o: bind
device: ${COMPOSE_PROJECT_DIR}/shared-uploads
mongo_data:
13 changes: 1 addition & 12 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,10 @@ services:
volume-init:
condition: service_completed_successfully
volumes:
# - ./frontend:/app:delegated
# - /app/node_modules
# - /app/dist
- ./.env:/app/.env:ro
environment:
NODE_ENV: ${NODE_ENV:-production}
CI: "true" # "true" # comment out when in development
CHOKIDAR_USEPOLLING: "false" ## "false"
WATCHPACK_POLLING: "false" ## "false"
CI: "true"

backend:
build:
Expand All @@ -35,9 +30,6 @@ services:
volume-init:
condition: service_completed_successfully
volumes:
# - ./backend:/app
# - /app/node_modules
# - /app/dist
- ./.env:/app/.env
- logs:/app/logs
- shared-uploads:/app/shared-uploads
Expand All @@ -60,13 +52,10 @@ services:
ports:
- "${WORKFLOW_PORT}:${WORKFLOW_PORT_DOCKER}"
volumes:
# - ./workflow:/app
- /var/run/docker.sock:/var/run/docker.sock
- logs:/app/logs
- shared-uploads:/app/shared-uploads:ro
depends_on:
# backend:
# condition: service_healthy
volume-init:
condition: service_completed_successfully
python_image:
Expand Down
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject",
"start:prod": "serve -s build"
"start:prod": "serve -s build -l ${FRONTEND_PORT_DOCKER:-4000}"
},
"eslintConfig": {
"extends": [
Expand Down
Loading

0 comments on commit 8949c3c

Please sign in to comment.