Using dynamic extractor with index in flow

[pascal-sun]

Nuclei version:

Nuclei Engine Version: v3.0.3

Current Behavior:

I want to use extractor as a dynamic variable in a flow (for example flow: http(1) && http(2), to pass the extracted value from http(1) and give it to http(2)).
When I extract using regex, it gives me always a list (test => [XXXX] from variables dump), even if I specify group: 1 for example.

Expected Behavior:

Should give me the value, as I specify the group: N(something like test => XXXX)
.

Steps To Reproduce:

Example of template:

id: extracted_value

info:
  name: Extracted value does not work in flow
  author: pascal-sun
  severity: info

flow: http(1) && http(2)

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    extractors: 
      - type: regex
        name: test
        group: 1
        regex:
          - 'This (domain) is for use in illustrative examples in documents.'
        internal: true

    matchers:
      - type: status
        status:
          - 200
 
  - method: GET
    path:
      - "https://example.org/{{test}}"
    matchers:
      - type: status
        status:
          - 200

It extracts correctly the variable test, but as list. And when I tried to use it in the second http, it stays as list

# nuclei -u https://example.com -t test.yaml -debug -svd

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.0.3

		projectdiscovery.io

[INF] Current nuclei version: v3.0.3 (latest)
[INF] Current nuclei-templates version: v9.6.9 (latest)
[INF] New templates added in latest release: 73
[INF] Templates loaded for current scan: 1
[WRN] Executing 1 unsigned templates. Use with caution.
[INF] Targets loaded for current scan: 1

[...]

[DBG] [extracted_value] Dumped HTTP response https://example.com

[...]

[DBG] HTTP Protocol request variables: 

	38. test => [domain]

[INF] [extracted_value] Dumped HTTP request for https://example.org/[domain]

Here, I want to send a request to https://example.org/domain and not https://example.org/[domain].

And when I remove the line for the flow (flow: http(1) && http(2)), the extractor works correctly.

Am I using it badly to extract variable in a flow, or is group: N no longer available in a flow?

[tovask]

Hi @pascal-sun,
I think here is a solution to your problem:

id: extracted_value

info:
  name: Extracted value does not work in flow
  author: pascal-sun
  severity: info

flow: |
  http(1);
  set("test2", template["test1"][0]);
  http(2);

http:
  - method: GET
    path:
      - "{{BaseURL}}"

    extractors: 
      - type: regex
        name: test1
        group: 1
        regex:
          - 'This (domain) is for use in illustrative examples in documents.'
        internal: true

    matchers:
      - type: status
        status:
          - 200
 
  - method: GET
    path:
      - "https://example.org/{{test2}}"
    matchers:
      - type: status
        status:
          - 200

Seems that when running from flow, the extractor put the extracted value(s) into a list (perhaps because it may match more than once in certain scenarios), and you have to take care of unpacking them.
(So it's not about the regex grouping, that seems to work fine.)

[ehsandeep]

@tovask thank you for sharing this!