Trouble Capturing MP3 with CDP Mode Across Multi-Step Flow

[benfrankie]

Hi,

I'm using SeleniumBase with uc=True and activate_cdp_mode() to automate a CAPTCHA flow from Captcha F​o​x.

On single-step versions, CDP successfully intercepts the .mp3 audio file.

But in the multi-step version (e.g. signup form at m a i l . c o m → checkbox → audio challenge → play), the page doesn’t reload or change URL — only the content updates dynamically. In these cases, CDP fails to capture the MP3 request when audio is played.

Is there a recommended way to ensure CDP captures dynamically triggered requests in multi-step flows like this, where the audio element appears only after several UI interactions on the same page?

Appreciate your help!

[SSujitX]

Appreciate your help!

Can you share your code?

[benfrankie]

Below is the outline of the code I tried.

from seleniumbase import SB

def test_audio_cdp():
with SB(uc=True, headless=False) as sb:

    current = sb.get_current_url()
    sb.activate_cdp_mode(current) # Also tried activating this later — both fail
    sb.driver.execute_cdp_cmd("Network.enable", {}) # this results in urllib3.exceptions.NewConnectionError
    sb.driver.execute_cdp_cmd("Page.enable",   {})
	
    mp3_urls = []
    async def on_req(evt: mycdp.network.RequestWillBeSent):
        url = evt.request.url
        if url.endswith(".mp3") and url not in mp3_urls:
            mp3_urls.append(url)
            sb.logger.info(f"✯ MP3 candidate: {url}")

    sb.cdp.add_handler(mycdp.network.RequestWillBeSent, on_req)

    # Go to target page 
    sb.uc_open_with_reconnect("https://example.com/signup")
	# Code for filling out the form here
	...
	solve_audio_captcha(sb, mp3_urls):
	...

def solve_audio_captcha(sb: SB, mp3_urls, max_retries=3) -> bool:

try:
    sb.wait_for_element('div[role="checkbox"]', timeout=10)
    sb.click('div[role="checkbox"]')
    sb.wait_for_element('button[aria-label="Switch to the audio challenge"]', timeout=10)
    sb.click('button[aria-label="Switch to the audio challenge"]')
    time.sleep(2)
except Exception as e:
    logger.error(f"❌ Error during initial CAPTCHA interaction: {e}")
    return False
...

for attempt in range(max_retries):
    if shutdown_event.is_set():
        logger.warning("Shutdown event triggered.")
        return

    logger.info(f"🔁 Attempt {attempt+1}/{max_retries} to solve audio CAPTCHA")

    try:
        sb.wait_for_element("button.cf-audio__trigger", timeout=10)
        sb.click("button.cf-audio__trigger")
        time.sleep(4)
    except Exception as e:
        logger.warning(f"❌ Failed to trigger audio playback: {e}")
        continue


    # 4️⃣  **New:** pull the <audio> or <source> src directly out of the page
    audio_urls = sb.execute_script("""
        return Array.from(document.querySelectorAll('audio'))
            .map(a => a.src || (a.querySelector('source') && a.querySelector('source').src))
            .filter(src => src && src.endsWith('.mp3'));
    """)
    if not audio_urls:
        logger.warning("❌ No <audio>/.mp3 URL found in DOM; retrying...")
        continue
    mp3_url = audio_urls[0]

    logger.info(f"🎯 Got MP3 URL: {mp3_url}")
    
    # First try reading from our mp3_urls list (handler)
    if mp3_urls:
        mp3_url = mp3_urls[-1]
    else:
        # fallback: scan browser console for our “CAPTCHA_AUDIO:” logs
        logs = sb.driver.get_log("browser")
        mp3_url = None
        for entry in logs:
            m = re.search(r"CAPTCHA_AUDIO:(https?://\S+\.mp3\S*)", entry["message"])
            if m:
                mp3_url = m.group(1)
                break

    if not mp3_url:
        logger.warning("❌ No MP3 URL detected; retrying...")
        continue

    # download & transcribe
    try:
        resp = requests.get(mp3_url, timeout=10)
        resp.raise_for_status()
        with tempfile.NamedTemporaryFile(suffix=".mp3", delete=False) as f:
            f.write(resp.content)
            audio_path = f.name
    except Exception as dlex:
        logger.error(f"❌ Failed to download audio: {dlex}")
        continue

I tried enabling activate_cdp_mode() both before navigating to the page and also right before clicking the audio challenge, but in the latter case it causes the form to reset or CAPTCHA to disappear.

Please advise. Thank you.

[benfrankie]

Any word on this? @mdmintz
Thanks in advance!

[mdmintz]

I don't know much about audio files, but I can tell you that sb.driver.execute_cdp_cmd will call the WebDriver API, which is detectable. Stick with the CDP Mode methods found at the bottom of examples/cdp_mode/ReadMe.md.

[benfrankie]

The .mp3 file is loaded via a dynamic request (e.g., cdn.captchafox.com/assets/audio/...mp3?...), and I need to intercept that request to download the audio and transcribe it.

I tried:
sb.driver.execute_cdp_cmd("Network.enable", {})
sb.driver.add_cdp_listener("Network.requestWillBeSent", callback)

but after sb.activate_cdp_mode(url), I get:

ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

I believe that’s because activate_cdp_mode() replaces/disconnects the original driver and switches to a headless CDP backend.

However, sb.cdp does not expose on(...) or any way to attach CDP listeners. Without a way to listen to Network.requestWillBeSent, I can't capture the MP3 dynamically during playback.

Could you consider adding support for CDP event listeners in sb.cdp, or suggest a workaround for intercepting .mp3 requests in CDP mode?

Thank you @mdmintz

[mdmintz]

There are lots of examples of capturing requests / responses:

• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_req_sb.py
• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_res_nike.py
• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_xhr_sb.py