refactor: remove models-store in favor of default ~/.docker/models#730
refactor: remove models-store in favor of default ~/.docker/models#730doringeman wants to merge 1 commit intodocker:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request aims to remove the local models-store in favor of the default ~/.docker/models path used by Docker Desktop. The changes in .gitignore, cmd/cli/Dockerfile, and demos/embeddings/indexer.js are correct and align with this goal. However, the modification to the Makefile breaks the docker-run functionality by removing the volume mount for the model store. I've provided a suggestion to fix this by using a default path for MODELS_PATH in the docker-run-impl target, ensuring that Docker-based runs continue to work as expected while adopting the new default model path.
| @echo "Starting service on port $(PORT)..." | ||
| @echo "Service will be available at: http://localhost:$(PORT)" | ||
| @echo "Example usage: curl http://localhost:$(PORT)/models" | ||
| @echo "" | ||
| PORT="$(PORT)" \ | ||
| MODELS_PATH="$(MODELS_PATH)" \ | ||
| DOCKER_IMAGE="$(DOCKER_IMAGE)" \ | ||
| LLAMA_ARGS="$(LLAMA_ARGS)" \ | ||
| DMR_ORIGINS="$(DMR_ORIGINS)" \ |
There was a problem hiding this comment.
These changes break the docker-run family of targets. While the Go application may have a default model path for local runs, when running in Docker, a volume must be mounted from the host to the container to share the model store. By removing MODELS_PATH from this rule, docker-run.sh will no longer mount any volume for models, and the container will be isolated from the host's ~/.docker/models directory.
To fix this while still defaulting to ~/.docker/models, you can use shell parameter expansion to provide a default value. This also keeps the ability for users to override the path via the command line (e.g., make docker-run MODELS_PATH=/custom/path).
@echo "Starting service on port $(PORT) with model storage at $${MODELS_PATH:-$${HOME}/.docker/models}..."
@echo "Service will be available at: http://localhost:$(PORT)"
@echo "Example usage: curl http://localhost:$(PORT)/models"
PORT="$(PORT)" \
MODELS_PATH="$${MODELS_PATH:-$${HOME}/.docker/models}" \
DOCKER_IMAGE="$(DOCKER_IMAGE)" \
LLAMA_ARGS="$(LLAMA_ARGS)" \
DMR_ORIGINS="$(DMR_ORIGINS)" \
762dd9c to
4087d46
Compare
The Go code already defaults MODELS_PATH to ~/.docker/models, so the Makefile override to a local models-store/ directory was unnecessary. This shares the model store with Docker Desktop's Model Runner. Signed-off-by: Dorin Geman <dorin.geman@docker.com>
4087d46 to
3731f27
Compare
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In scripts/docker-run.sh,
add_optional_argsnow relies onmodels_pathbutmodels_pathis declaredlocalinmain, so it won’t be visible insideadd_optional_argsin bash; either definemodels_pathat the top level (notlocal) or pass it explicitly intoadd_optional_argsto avoid mounting an empty path.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In scripts/docker-run.sh, `add_optional_args` now relies on `models_path` but `models_path` is declared `local` in `main`, so it won’t be visible inside `add_optional_args` in bash; either define `models_path` at the top level (not `local`) or pass it explicitly into `add_optional_args` to avoid mounting an empty path.
## Individual Comments
### Comment 1
<location path="scripts/docker-run.sh" line_range="66" />
<code_context>
main() {
set -eux -o pipefail
+ local models_path="${MODELS_PATH:-$HOME/.docker/models}"
local args=(docker run --rm -e LLAMA_SERVER_PATH=/app/bin)
add_optional_args
</code_context>
<issue_to_address>
**question (bug_risk):** The new defaulting behavior removes the ability to opt out of a models volume by setting an empty MODELS_PATH.
`${MODELS_PATH:-$HOME/.docker/models}` now mounts the volume even when `MODELS_PATH` is empty, falling back to `$HOME/.docker/models`. If you want to preserve the ability to disable the mount via an empty `MODELS_PATH`, you could use `${MODELS_PATH:-}` and only mount when it’s non-empty, or add an explicit flag to disable the models volume. If this change is intentional, it’s worth calling out as a behavioral change.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| main() { | ||
| set -eux -o pipefail | ||
|
|
||
| local models_path="${MODELS_PATH:-$HOME/.docker/models}" |
There was a problem hiding this comment.
question (bug_risk): The new defaulting behavior removes the ability to opt out of a models volume by setting an empty MODELS_PATH.
${MODELS_PATH:-$HOME/.docker/models} now mounts the volume even when MODELS_PATH is empty, falling back to $HOME/.docker/models. If you want to preserve the ability to disable the mount via an empty MODELS_PATH, you could use ${MODELS_PATH:-} and only mount when it’s non-empty, or add an explicit flag to disable the models volume. If this change is intentional, it’s worth calling out as a behavioral change.
The Go code already defaults MODELS_PATH to ~/.docker/models, so the Makefile override to a local models-store/ directory was unnecessary. This shares the model store with Docker Desktop's Model Runner.