Skip to content

Extend evaluator with render mode control#331

Merged
daphne-cornelisse merged 5 commits into2.0from
dc/extend_render_modes
Mar 9, 2026
Merged

Extend evaluator with render mode control#331
daphne-cornelisse merged 5 commits into2.0from
dc/extend_render_modes

Conversation

@daphne-cornelisse
Copy link

@daphne-cornelisse daphne-cornelisse commented Mar 9, 2026

Problem

It can be useful to render based on a particular criterion. For example, instead of always displaying the first env, you may want to continually show the worst scenario in a batch of scenes. This was not possible because vec_log(), which is used to collect stats from the envs, aggregates across agents and envs, losing per-env granularity.

Solution

  • Added env_log() C binding that returns statistics per environment without aggregating across envs.
  • Added configurable render selection modes (first, random, worst_score, worst_collision) via render_select_mode in the eval config.
  • For worst-case modes, the Evaluator runs a stats-only rollout first, selects the target env using env_log, then renders that env in a second rollout.

Pipeline

Before

Before, the eval pipeline looked like this:

Screenshot 2026-03-09 at 10 35 31 AM

After

Now, if the render mode is not fixed nor random, it looks like this:

Screenshot 2026-03-09 at 10 35 38 AM

Logging example

Screenshot 2026-03-09 at 4 30 37 PM

@daphne-cornelisse daphne-cornelisse marked this pull request as ready for review March 9, 2026 19:57
@daphne-cornelisse daphne-cornelisse changed the title Extend evaluator with render mode control. Extend evaluator with render mode control Mar 9, 2026
@daphne-cornelisse daphne-cornelisse added the enhancement New feature or request label Mar 9, 2026
@daphne-cornelisse daphne-cornelisse merged commit 02f1008 into 2.0 Mar 9, 2026
13 checks passed
@daphne-cornelisse daphne-cornelisse deleted the dc/extend_render_modes branch March 9, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant