-
Notifications
You must be signed in to change notification settings - Fork 260
Description
Configuring a common model cache location for the CLI and the C# SDK results in an Internal Server Error.
C:\Users\User>foundry cache ls
Exception: Failed: Getting list of downloaded models.
Response status code does not indicate success: 500 (Internal Server Error).
More details:
I only started looking at Local Foundry recently. First, I tested the CLI:
-
Downloaded and ran couple of models (
Phi-4-mini-instructandPhi-4-mini-reasoning). -
Listing the cache showed both models
Next, I tested the C# SDK by running the HelloFoundryLocalSdk project unchanged from the FoundrySamplesWinML solution in this repo.
-
The sample downloaded the
qwen2.5-0.5bmodel and executed successfully. However, when I ranfoundry cache lsexpecting it to also include the newly downloaded model, I was surprised to see that it only listed the models I'd previously downloaded with the CLI. After some investigation, I found out that the C# SDK and the CLI don't necessarily share the same cache folder by default so it had cached the model at.foundry_local_samplesin my user profile. I deleted this folder, updated the code as follows to set the cache location to be the same as the CLI, and then ran the sample again. It now downloaded the model to the shared cache location, as expected.var config = new Configuration { AppName = "foundry_local_samples", ModelCacheDir = "E:\\AI\\Foundry\\Models", // added this line LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information };
-
Next, I ran
foundry cache lsand got the following error message. Restarting the service made no difference.
-
After some troubleshooting, I discovered that downloading a new model with the CLI "fixed" the problem and allowed listing the cache again. However, it broke again as soon as I ran a sample from the C# SDK.
-
Upon further investigation, I noticed that the
foundry.modelinfo.jsonfile found at the cache location and rewritten after downloading a model with the CLI or when executing a C# SDK sample have different formats, which presumably is related to the behavior I'm seeing.
-
I renamed the
foundry.modelinfo.jsonfile and then ranfoundry cache lsagain. It rewrote a newfoundry.modelinfo.jsonfile and the error disappeared.
Note that the additional models in the cache are the result of other tests I carried out while troubleshooting and experimenting with Foundry since I first noticed this issue.
It might also be related to this issue (#411).
