Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does cats support single interface testing #139

Open
duhang1996 opened this issue Sep 26, 2024 · 8 comments
Open

Does cats support single interface testing #139

duhang1996 opened this issue Sep 26, 2024 · 8 comments

Comments

@duhang1996
Copy link

Does cats support single interface testing?
such as:
https://cloud.githubusercontent.com/assets/4562611/19390428/12224610-91f6-11e6-9ece-6e3cd7dd35ea.png

@duhang1996
Copy link
Author

java -jar target/cats-runner.jar fuzz -X GET --mc 500 -t "path1" -i "2XX,4XX" https://petstore3.swagger.io/api/v3/pet/path1

[][] ▶ Starting cats-11.8.1-SNAPSHOT, build time 2024-09-26T06:32:26Z UTC, platform Mac OS X-12.0-aarch64

================================================================================= FUZZING =================================================================================

https://petstore3.swagger.io/api/v3/pet/path1 .............................................................................................................. E 0, W 0, S 0 ✔
[][] ℹ Skip printing time execution statistics. You can use --printExecutionStatistics to enable this feature!

[][] ★ CATS finished in 14.749s. Total requests 0. ✔ Passed 0, ⚠ warnings: 0, ‼ errors: 0, You can open the report here:

Why there were 0 tests generated?Help

@en-milie
Copy link
Contributor

Hi @duhang1996. Yes it does. Exactly as you executed. The report has zero tests because no test matched 500 as you instructed. If you run with --mc 400 you will get all tests in the report. If you want to see the execution details, you can also include --verbosity DETAILED as an argument.

@duhang1996
Copy link
Author

Hi @duhang1996. Yes it does. Exactly as you executed. The report has zero tests because no test matched 500 as you instructed. If you run with --mc 400 you will get all tests in the report. If you want to see the execution details, you can also include --verbosity DETAILED as an argument.

thank you!

@duhang1996
Copy link
Author

duhang1996 commented Sep 29, 2024

Hi @duhang1996. Yes it does. Exactly as you executed. The report has zero tests because no test matched 500 as you instructed. If you run with --mc 400 you will get all tests in the report. If you want to see the execution details, you can also include --verbosity DETAILED as an argument.

Hi, @en-milie After several rounds of testing, I found that the generated test is the same every time. How to keep this test constantly running or generate more test cases?Thanks!

@en-milie
Copy link
Contributor

When you say same tests what do you mean? As a scenario? What CATS does with the built in mechanism is to send a predefined set of random characters in the fields you specify. For more intelligence you should use an OpenAPI spec. That will create more advanced scenarios.
You can also provide your own dictionary using --words <file>:

java -jar target/cats-runner.jar fuzz -X GET --mc 400 -t "path1" -i "2XX,4XX" --words nastyStrings.txt https://petstore3.swagger.io/api/v3/pet/path1

@duhang1996
Copy link
Author

@en-milie What I mean is that the random replacement characters generated for the first and second times are exactly the same, and the number of test cases generated is also the same(38).After executing the 38 test cases, the run ends and cannot be tested continuously.
the first test:
image
image

the second test:
image

@en-milie
Copy link
Contributor

Do you have an OpenAPI spec? Or just fuzzing based on a JSON template?

@en-milie
Copy link
Contributor

en-milie commented Nov 9, 2024

Starting with this release https://github.com/Endava/cats/releases/tag/cats-13.0.0 you can add an additional --random argument to the cats fuzz sub-command that will do continuous fuzzing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants