Skip to content

Commit

Permalink
Solve issues - #247, #246, #245, #244, #243
Browse files Browse the repository at this point in the history
  • Loading branch information
@LongquanJiang
LongquanJiang committed Jun 20, 2024
1 parent 6f15a71 commit f0cb47b
Show file tree
Hide file tree
Showing 5 changed files with 171 additions and 125 deletions.
46 changes: 28 additions & 18 deletions freebase/ComplexWebQuestions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@

| Model / System | Year | F1 | Hits@1 | Accuracy | Language | Reported by |
|:------------------------:|:----:|:----:|:----------:|----------:|:--------:|:---------------------------------------------------------------------------:|
| GMT-KBQA | 2022 | 77.0 | - | 72.2 | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| CBR-KBQA | 2022 | 70.0 | - | 67.1 | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| chatGPT | 2023 | - | - | 64.02 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3.5v3 | 2023 | - | - | 57.54 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3.5v2 | 2023 | - | - | 53.96 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3 | 2023 | - | - | 51.77 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| FLAN-T5 | 2023 | - | - | 46.69 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| BART-large | 2022 | 68.2 | - | - | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| GMT-KBQA | 2022 | 77.0 | - | 72.2 | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| CBR-KBQA | 2022 | 70.0 | - | 67.1 | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| chatGPT | 2023 | - | - | 64.02 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3.5v3 | 2023 | - | - | 57.54 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3.5v2 | 2023 | - | - | 53.96 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| GPT-3 | 2023 | - | - | 51.77 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| FLAN-T5 | 2023 | - | - | 46.69 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| BART-large | 2022 | 68.2 | - | - | EN | [Hu et al.](https://aclanthology.org/2022.coling-1.145.pdf) |
| DECAF (BM25 + FiD-3B) | 2022 | - | 70.4 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| CBR-KBQA | 2022 | - | 70.4 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| DECAF (BM25 + FiD-large) | 2022 | - | 68.1 ± 0.5 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
Expand All @@ -22,15 +22,15 @@
| ProgramTransfer-o | 2022 | 55.8 | 54.7 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| ProgramTransfer-pa | 2022 | 54.5 | 54.3 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| NSM+h | 2022 | - | 53.9 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| REAREV | 2022 | - | 52.9 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| REAREV | 2022 | - | 52.9 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| QNRKGQA+h | 2022 | - | 51.5 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| SR+NSM | 2022 | - | 50.2 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| QNRKGQA | 2022 | - | 50.5 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| NSM-distill | 2022 | - | 48.8 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| Rigel | 2022 | - | 48.7 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| TransferNet | 2022 | - | 48.6 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| PullNet | 2022 | - | 45.9 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| MRP-QA-marginal_prob | 2022 | 49.9 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| NSM-distill | 2022 | - | 48.8 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| Rigel | 2022 | - | 48.7 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| TransferNet | 2022 | - | 48.6 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| PullNet | 2022 | - | 45.9 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| MRP-QA-marginal_prob | 2022 | 49.9 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| ProgramTransfer-f | 2022 | 45.9 | 45.2 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| KBIGER | 2022 | 45.5 | 50.2 | - | EN | [Du et al.](https://arxiv.org/pdf/2209.03005.pdf) |
| TERP | 2022 | - | 49.2 | - | EN | [Qiao et al.](https://aclanthology.org/2022.coling-1.156.pdf) |
Expand All @@ -42,15 +42,25 @@
| NSM | 2022 | - | 47.6 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| PullNet | 2019 | - | 47.2 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| PullNet | 2022 | - | 45.9 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| EmbedKGQA | 2022 | - | 44.7 | - | EN | [Qiao et al.](https://aclanthology.org/2022.coling-1.156.pdf) |
| EmbedKGQA | 2022 | - | 44.7 | - | EN | [Qiao et al.](https://aclanthology.org/2022.coling-1.156.pdf) |
| QGG | 2020 | 40.4 | 44.1 | - | EN | [Lan and Jiang et al.](https://aclanthology.org/2020.acl-main.91.pdf) |
| Topic Units | 2019 | 36.5 | - | - | EN | [Lan et al.](https://www.ijcai.org/proceedings/2019/0701.pdf) |
| KBQA-GST | 2022 | 36.5 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| KBQA-GST | 2022 | 36.5 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| TextRay | 2019 | 33.9 | 40.8 | - | EN | [Bhutani et al.](https://dl.acm.org/doi/10.1145/3357384.3358033) |
| HR-BiLSTM | 2022 | 31.2 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| HR-BiLSTM | 2022 | 31.2 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| KGT5 | 2019 | - | 36.5 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
| GraphNet | 2022 | - | 32.8 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| KV-Mem | 2022 | - | 21.1 | - | EN | [Ma et al.](https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11) |
| UHop | 2019 | 29.8 | - | - | EN | [Chen et al.](https://arxiv.org/pdf/1904.01246.pdf) |
| GRAFT-Net | 2022 | 26.0 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| GRAFT-Net | 2022 | 26.0 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| ProgramTransfer-p | 2022 | 2.3 | 2.1 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| GoG w/GPT-3.5 | 2024 | - | 55.7 | - | EN | [Xu et al.](https://arxiv.org/pdf/2404.14741) |
| GoG w/GPT-4 | 2024 | - | 75.2 | - | EN | [Xu et al.](https://arxiv.org/pdf/2404.14741) |
| IO prompt w/ChatGPT | 2024 | - | 37.6 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| CoT prompt w/ChatGPT | 2024 | - | 38.8 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| SC prompt w/ChatGPT | 2024 | - | 45.4 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| Prior FT SOTA | 2024 | - | 70.4 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| EffiQA w/ChatGPT | 2024 | - | 52.1 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| EffiQA w/Deepseek-V2 | 2024 | - | 61.7 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| EffiQA w/GPT-4 | 2024 | - | 69.5 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
| Prior tigh-coupling SOTA | 2024 | - | 72.5 | - | - | EN | [Dong et al.](https://arxiv.org/pdf/2406.01238) |
Loading

0 comments on commit f0cb47b

Please sign in to comment.