How can I set the AdamW optimizer?

[LyoungOVO]

When I set the batch size to 8 in SwinT and Convnext ,the ap was down.The original bs is 2.How can I set the AdamW and learning policy?I look forward to your reply,thanks.

[Ezra-Yu]

Can show your config and the version used？

[LyoungOVO]

TorchVision: 0.11.1
OpenCV: 4.6.0
MMCV: 1.5.0
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.3
MMRotate: 0.2.0
MMCls:0.23.2

optimizer

optimizer = dict(
delete=True,
type='AdamW',
lr=0.0001,
betas=(0.9, 0.999),
weight_decay=0.05,
paramwise_cfg=dict(
custom_keys={
'absolute_pos_embed': dict(decay_mult=0.),
'relative_position_bias_table': dict(decay_mult=0.),
'norm': dict(decay_mult=0.)
}))
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))

learning policy

lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=1.0 / 3,
step=[8, 11])
runner = dict(type='EpochBasedRunner', max_epochs=12)

[Ezra-Yu]

If you use 1.x branch, you can use --auto-scale-lr after setting base_batch_size

[LyoungOVO]

Sorry,I am using the MMRotate,it doesn't support --auto-scale-lr. Can you tell me where is the --auto-scale-lr code,I will set the lr according to it,thanks.

[Ezra-Yu]

Sorry that --auto-scale-lr only supported in the 1.x branch. just add a auto_scale_lr = dict(enable=True, base_batch_size=16) in config file.

[Ezra-Yu]

in mmengine: https://github.com/open-mmlab/mmengine/blob/64ac14303fc786151beabba1de4ddfd7c2e5a408/mmengine/runner/runner.py#L894

[LyoungOVO]

Ok,thanks!