Question on Normalization Methods for Non-Atari Benchmarks in Rliable Analysis

[amantuer]

Hello Rliable team,

I’m using Rliable for analyzing reinforcement learning results on environments such as the DeepMind Control Suite and PyBullet, where human-normalized scores (as used in Atari benchmarks) are unavailable for recent algorithms like SARC. I noticed that in Atari experiments, human benchmarks are crucial to standardize scores across algorithms. Without equivalent baselines in DMC or PyBullet, I’m considering Z-score normalization and percentile normalization for comparing different RL algorithms.

Could you share any guidance on best practices for normalization in these cases or suggest other robust approaches that align with Rliable’s statistical rigor?

Thank you for the insightful tools and methodologies you provide!