Questions about performance comparison in the paper

[garyzhang99]

Hi team,

Just read your paper and loved it!

I'm really interested in the specific comparison of effects you made. The paper mainly compares methods like DSPy(BFSR) and Reflection, but there isn’t much on how other prompt optimization methods stack up against these.

I'm curious about how the textgrad method performs compared to other prompt optimization methods like APE, PE2, and APO. Particularly, you mentioned the Prompt Optimization with Textual Gradients (ProTeGi) method in your paper. How does textgrad compare to ProTeGi in terms of performance? Does it offer any further performance gains?
Additionally, are these methods applicable to datasets beyond GSM8K and BigBench, or do they have limitations in other contexts while textgrad don't?

If you've done any experiments or have insights on these comparisons, could you share your results or thoughts?

Thanks a lot for your amazing work!