Skip to content

Issue configuring a common cache location between the CLI and C# SDK #442

@f2bo

Description

@f2bo

Configuring a common model cache location for the CLI and the C# SDK results in an Internal Server Error.

C:\Users\User>foundry cache ls
Exception: Failed: Getting list of downloaded models.
Response status code does not indicate success: 500 (Internal Server Error).

More details:

I only started looking at Local Foundry recently. First, I tested the CLI:

  • Changed the CLI's cache location to E:\AI\Foundry\Models.
    Image

  • Downloaded and ran couple of models (Phi-4-mini-instruct and Phi-4-mini-reasoning).

  • Listing the cache showed both models

    Image

Next, I tested the C# SDK by running the HelloFoundryLocalSdk project unchanged from the FoundrySamplesWinML solution in this repo.

  • The sample downloaded the qwen2.5-0.5b model and executed successfully. However, when I ran foundry cache ls expecting it to also include the newly downloaded model, I was surprised to see that it only listed the models I'd previously downloaded with the CLI. After some investigation, I found out that the C# SDK and the CLI don't necessarily share the same cache folder by default so it had cached the model at .foundry_local_samples in my user profile. I deleted this folder, updated the code as follows to set the cache location to be the same as the CLI, and then ran the sample again. It now downloaded the model to the shared cache location, as expected.

    var config = new Configuration
    {
        AppName = "foundry_local_samples",
        ModelCacheDir = "E:\\AI\\Foundry\\Models",			// added this line
        LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
    };
  • Next, I ran foundry cache ls and got the following error message. Restarting the service made no difference.

    Image
  • After some troubleshooting, I discovered that downloading a new model with the CLI "fixed" the problem and allowed listing the cache again. However, it broke again as soon as I ran a sample from the C# SDK.

  • Upon further investigation, I noticed that the foundry.modelinfo.json file found at the cache location and rewritten after downloading a model with the CLI or when executing a C# SDK sample have different formats, which presumably is related to the behavior I'm seeing.

Image
  • I renamed the foundry.modelinfo.json file and then ran foundry cache ls again. It rewrote a new foundry.modelinfo.json file and the error disappeared.

    Image Note that the additional models in the cache are the result of other tests I carried out while troubleshooting and experimenting with Foundry since I first noticed this issue.

It might also be related to this issue (#411).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions