Revert "Pin conda build images - fix nightly conda failures (#4582)"

[atalman]

This

1. reverts commit 389b0ab.
2. fixes issue with conda picking up default test env created by: https://github.com/conda-incubator/setup-miniconda#conda-incubatorsetup-miniconda
   That forced us to do docker pin.
   Here is the failure:
   https://github.com/pytorch/test-infra/actions/runs/6474243290/job/17578773829?pr=4606#step:10:1113

Root Cause: conda is picking test environment by default rather then $CONDA_ENV for building the package

++++ __conda_exe shell.posix activate base
++++ /opt/conda/bin/conda shell.posix activate base
+++ ask_conda='export PATH='\''/opt/conda/conda-bld/torchvision_1696968398076/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh:/opt/conda/conda-bld/torchvision_1696968398076/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/bin:/opt/conda/bin:/opt/conda/condabin:3/condabin:/opt/conda/bin:/opt/rh/devtoolset-9/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'\''

3. Removes python 3.11 case, since conda is fully supporting python 3.11 now

[vercel]

@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

[malfet]

Sounds OK, but would be nice to have a more detailed write-up explaining why pining was necessary and why we want to remove it right now. Isn't it better to keep the build system pinned.

@atalman: Point 2 was the cause of pinning. For release builds I agree we should use pinned images. For build system not so sure since all our CI depend on these docker builds that are updated regularly. If we stop using latest images it maybe harder to update later.

[henrylhtsang]

@atalman I think this pr is causing some torchrec python 3.11 build-linux-wheels tests to fail.

11 hours, tests passing
https://github.com/pytorch/torchrec/actions/runs/6483145174

6 hours ago, python 3.11 failing
https://github.com/pytorch/torchrec/actions/runs/6486263265

Also, when I run conda create -n test --yes --quiet python=3.11 cmake=3.22 conda-build=3.21 ninja=1.10 pkg-config=0.29 wheel=0.37 locally, it gives similar errors.