Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sweep: figure out a better solution to do https://github.com/beetcb/ghdl/commit/1f17aa96e33c912de7557612df3e628d5f91ebaf #3

Closed
beetcb opened this issue Jun 2, 2022 · 12 comments · May be fixed by #7
Closed

sweep: figure out a better solution to do https://github.com/beetcb/ghdl/commit/1f17aa96e33c912de7557612df3e628d5f91ebaf #3

beetcb opened this issue Jun 2, 2022 · 12 comments · May be fixed by #7
Labels
sweep Assigns Sweep to an issue or pull request.

Comments

@beetcb
Copy link
Owner

beetcb commented Jun 2, 2022

ghdl/dl.go

Line 72 in f53255a

// Package format `deb` `rpm` `apk` will be downloaded directly

@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023 •
edited
Loading

And .dmg, and other possible os-specific binary file extensions as well

@beetcb beetcb added the sweep Assigns Sweep to an issue or pull request. label Aug 5, 2023
Repository owner deleted a comment from sweep-ai bot Aug 5, 2023
@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023

sweep: read this issue

@sweep-ai
Copy link
Contributor

sweep-ai bot commented Aug 5, 2023 •
edited
Loading

Here's the PR! #7.

⚡ Sweep Free Trial: I used GPT-4 to create this ticket. You have 3 GPT-4 tickets left. For more GPT-4 tickets, visit our payment portal.To get Sweep to recreate this ticket, leave a comment prefixed with "sweep:" or edit the issue.

Step 1: 🔍 Code Search

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I looked at (click to expand). If some file is missing from here, you can mention the path in the ticket description.

ghdl/README.md

Lines 1 to 95 in 11c9c7c

# ghdl
> Memorize `ghdl` as `github download`
`ghdl` is a fast and simple program (and also a golang module) for downloading and installing executable binary from github releases.
<p align="center">
<img alt="animated demo" src="./demo.svg" width="600px">
</p>
<p align="center">
<strong>The demo above extracts <code>fd</code> execuable to current working directory and give execute permission to it.</strong>
</p>
# Features
- Auto decompressing and unarchiving the downloaded asset (without any system dependencies like `tar` or `unzip`)
```ts
Currently supporting unarchiving `tar` and decompressing `zip` `gzip`.
Package format `deb` `rpm` `apk` will be downloaded directly
```
`ghdl` binary is statically linked, works well on non-FHS *nix systems like [NixOS](https://nixos.org/)). In case this is relevant to you, on that kind of system, only binaries like `ghdl` can be run directly.
- Setups for executable: `ghdl` moves executable to specified location and add execute permissions to the file.
- Auto filtering: multiple assets in one release will be filtered by OS or ARCH. This feature can be disabled using `-F` flag.
- Interactive TUI: when auto filtering is failed or returned multiple options, you can select assets in a interactive way, with vim key bindings support.
- Release tags: `ghdl` downloads latest release by default, other or old tagged releases can be downloaded by specifying release tag: `username/repo#tagname`
- Inspect download status with real-time progress bar.
# Installation
> If you're going to use `ghdl` as a go module, ignore the following installation progress.
- Using Go tools:
go will download the latest version of ghdl to $GOPATH/bin, please make sure $GOPATH is in the PATH:
```sh
go install github.com/beetcb/ghdl/ghdl@latest
```
> Note: Just to be safe, you'd better specify CGO_ENABLED=0 when running `go install` on non-FHS *nix systems like [NixOS](https://nixos.org/))
- Download and run executable from release.
- Run the following shell script(*nix system only):
```sh
curl -fsSL "https://bina.egoist.sh/beetcb/ghdl?dir=/usr/local/bin" | sh
# feel free to change the `dir` url param to specify the installation directory.
```
# Usage
### CLI
Run `ghdl --help`
```sh
❯ ghdl --help
ghdl download binary from github release
ghdl handles archived or compressed file as well
Usage:
ghdl <user/repo[#tagname]> [flags]
Flags:
-f, --asset-filter string specify regular expression for the asset name; used in conjunction with the platform and architecture filters.
-F, --filter-off turn off auto-filtering feature
-h, --help help for ghdl
-n, --name string specify binary file name to enhance filtering and extracting accuracy
-p, --path path save binary to path and add execute permission to it (default ".")
```
It's tedious to specify `-p` manually, we can alias `ghdl -p "$DirInPath"` to a shorthand command, then use it as a executable installer.
### Go Module
1. Require `ghdl` to go.mod
```sh
go get github.com/beetcb/ghdl
```
2. Use `ghdl`'s out-of-box utilities: see [TestDownloadFdBinary func](./ghdl_test.go) as an example
# Credit
Inspired by [egoist/bina](https://github.com/egoist/bina), TUI powered by [charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea)
# License
Licensed under [MIT](./LICENSE)
Author: @beetcb | Email: i@beetcb.com

ghdl/helper/sl/sl.go

Lines 1 to 80 in 11c9c7c

package sl
import (
"fmt"
"os"
h "github.com/beetcb/ghdl/helper"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
)
type model struct {
choices []string // items on the to-do list
cursor int // which to-do list item our cursor is pointing at
selected int // which to-do items are selected
}
func initialModel(choices *[]string) model {
return model{
choices: *choices,
selected: -1,
}
}
func (m model) Init() tea.Cmd {
// Just return `nil`, which means "no I/O right now, please."
return nil
}
func (m *model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
switch msg := msg.(type) {
case tea.KeyMsg:
switch msg.String() {
case "up", "k":
if m.cursor > 0 {
m.cursor--
}
case "down", "j":
if m.cursor < len(m.choices)-1 {
m.cursor++
}
case "enter":
m.selected = m.cursor
return m, tea.Quit
}
}
return m, nil
}
func (m model) View() string {
blue, printWidth := lipgloss.Color("14"), 60
paddingS := lipgloss.NewStyle().PaddingLeft(2).Width(printWidth)
colorS := paddingS.Copy().
Foreground(blue).BorderLeft(true).BorderForeground(blue)
s := h.Sprint("multiple options after filtering, please select asset manually", h.SprintOptions{PrintWidth: 80}) + "\n"
if m.selected == -1 {
for i, choice := range m.choices {
if m.cursor == i {
s += colorS.Render(choice) + "\n"
} else {
s += paddingS.Render(choice) + "\n"
}
}
// Send the UI for rendering
return s + "\n"
} else {
return s
}
}
func Select(choices *[]string) int {
state := initialModel(choices)
p := tea.NewProgram(&state)
if err := p.Start(); err != nil {
h.Println(fmt.Sprintf("Alas, there's been an error: %v", err), h.PrintModeErr)
os.Exit(1)
}
return state.selected
}

ghdl/dl.go

Lines 1 to 178 in 11c9c7c

package ghdl
import (
"archive/tar"
"archive/zip"
"compress/gzip"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/beetcb/ghdl/helper/pg"
humanize "github.com/dustin/go-humanize"
)
var (
ErrNeedInstall = errors.New(
"detected deb/rpm/apk package, download directly")
ErrNoBin = errors.New("binary file not found")
)
type GHReleaseDl struct {
BinaryName string
Url string
Size int64
}
// Download asset from github release to `path`
//
// dl.BinaryName shall be replaced with absolute path mutably
func (dl *GHReleaseDl) DlTo(path string) (err error) {
dl.BinaryName, err = filepath.Abs(filepath.Join(path, dl.BinaryName))
if err != nil {
return err
}
req, err := http.NewRequest("GET", dl.Url, nil)
if err != nil {
return err
}
req.Header.Set("Accept", "application/vnd.github.v3+json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
tmpfile, err := os.Create(dl.BinaryName + ".tmp")
if err != nil {
return err
}
defer tmpfile.Close()
// create progress tui
starter := func(updater func(float64)) {
if _, err := io.Copy(tmpfile, &pg.ProgressBytesReader{Reader: resp.Body, Handler: func(p int) {
updater(float64(p) / float64(dl.Size))
}}); err != nil {
panic(err)
}
}
pg.Progress(starter, humanize.Bytes(uint64(dl.Size)))
return nil
}
// Extract binary file from the downloaded temporary file.
//
// Currently supporting unarchiving `tar` and decompressing `zip` `gravezip`.
//
// Package format `deb` `rpm` `apk` will be downloaded directly
func (dl GHReleaseDl) ExtractBinary() error {
tmpfileName := dl.BinaryName + ".tmp"
openfile, err := os.Open(tmpfileName)
if err != nil {
return err
}
fileExt := filepath.Ext(dl.Url)
var decompressedBinary io.Reader
switch fileExt {
case ".zip":
zipFile, err := dl.UnZipBinary(openfile)
if err != nil {
return err
}
decompressedBinary, err = zipFile.Open()
if err != nil {
return err
}
case ".gz":
if strings.Contains(dl.Url, ".tar.gz") {
decompressedBinary, err = dl.UnTargzBinary(openfile)
if err != nil {
return err
}
} else {
decompressedBinary, err = dl.UnGzBinary(openfile)
if err != nil {
return err
}
}
case "":
decompressedBinary = openfile
case ".deb", ".rpm", ".apk", ".msi", ".exe", ".dmg":
fileName := dl.BinaryName + fileExt
if err := os.Rename(tmpfileName, fileName); err != nil {
panic(err)
}
return ErrNeedInstall
default:
defer os.Remove(tmpfileName)
return fmt.Errorf("unsupported file format: %v", fileExt)
}
defer os.Remove(tmpfileName)
defer openfile.Close()
out, err := os.Create(dl.BinaryName)
if err != nil {
return err
}
defer out.Close()
if _, err := io.Copy(out, decompressedBinary); err != nil {
return err
}
return nil
}
func (dl GHReleaseDl) UnZipBinary(r *os.File) (*zip.File, error) {
b := filepath.Base(dl.BinaryName)
zipR, err := zip.NewReader(r, dl.Size)
if err != nil {
return nil, err
}
for _, f := range zipR.File {
if filepath.Base(f.Name) == b || len(zipR.File) == 1 {
return f, nil
}
}
return nil, ErrNoBin
}
func (GHReleaseDl) UnGzBinary(r *os.File) (*gzip.Reader, error) {
gzR, err := gzip.NewReader(r)
if err != nil {
return nil, err
}
defer gzR.Close()
return gzR, nil
}
func (dl GHReleaseDl) UnTargzBinary(r *os.File) (*tar.Reader, error) {
b := filepath.Base(dl.BinaryName)
gzR, err := gzip.NewReader(r)
if err != nil {
return nil, err
}
defer gzR.Close()
tarR := tar.NewReader(gzR)
for {
header, err := tarR.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
if (header.Typeflag != tar.TypeDir) && filepath.Base(header.Name) == b {
if err != nil {
return nil, err
}
return tarR, nil
}
}
return nil, ErrNoBin
}

ghdl/ghdl/main.go

Lines 1 to 108 in 11c9c7c

package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/beetcb/ghdl"
h "github.com/beetcb/ghdl/helper"
"github.com/spf13/cobra"
)
// rootCmd represents the base command when called without any subcommands
var rootCmd = &cobra.Command{
Use: "ghdl <user/repo[#tagname]>",
Short: "ghdl download binary from github release",
Long: `ghdl download binary from github release
ghdl handles archived or compressed file as well`,
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
cmdFlags := cmd.Flags()
binaryNameFlag, err := cmdFlags.GetString("name")
if err != nil {
panic(err)
}
pathFlag, err := cmdFlags.GetString("path")
if err != nil {
panic(err)
}
filterOff, err := cmdFlags.GetBool("filter-off")
if err != nil {
panic(err)
}
assetFilterString, err := cmdFlags.GetString("asset-filter")
if err != nil {
panic(err)
}
var assetFilter *regexp.Regexp
if assetFilterString != "" {
assetFilter, err = regexp.Compile(assetFilterString)
if err != nil {
panic(err)
}
}
repo, tag := parseArg(args[0])
ghRelease := ghdl.GHRelease{RepoPath: repo, TagName: tag}
ghReleaseDl, err := ghRelease.GetGHReleases(filterOff, assetFilter)
if err != nil {
h.Println(fmt.Sprintf("get gh releases failed: %s", err), h.PrintModeErr)
os.Exit(1)
}
if binaryNameFlag != "" {
ghReleaseDl.BinaryName = binaryNameFlag
}
h.Println(fmt.Sprintf("start downloading %s", h.Sprint(filepath.Base(ghReleaseDl.Url), h.SprintOptions{PromptOff: true, PrintMode: h.PrintModeSuccess})), h.PrintModeInfo)
if err := ghReleaseDl.DlTo(pathFlag); err != nil {
h.Println(fmt.Sprintf("download failed: %s", err), h.PrintModeErr)
os.Exit(1)
}
if err := ghReleaseDl.ExtractBinary(); err != nil {
switch err {
case ghdl.ErrNeedInstall:
h.Println(fmt.Sprintf("%s. You can install it with the appropriate commands", err), h.PrintModeInfo)
os.Exit(0)
case ghdl.ErrNoBin:
h.Println(fmt.Sprintf("%s. Try to specify binary name flag", err), h.PrintModeInfo)
os.Exit(0)
default:
h.Println(fmt.Sprintf("extract failed: %s", err), h.PrintModeErr)
os.Exit(1)
}
}
h.Println(fmt.Sprintf("saved executable to %s", ghReleaseDl.BinaryName), h.PrintModeSuccess)
if err := os.Chmod(ghReleaseDl.BinaryName, 0777); err != nil {
h.Println(fmt.Sprintf("chmod failed: %s", err), h.PrintModeErr)
}
},
}
func main() {
err := rootCmd.Execute()
if err != nil {
os.Exit(1)
}
}
func init() {
rootCmd.PersistentFlags().StringP("name", "n", "", "specify binary file name to enhance filtering and extracting accuracy")
rootCmd.PersistentFlags().StringP("asset-filter", "f", "",
"specify regular expression for the asset name; used in conjunction with the platform and architecture filters.")
rootCmd.PersistentFlags().StringP("path", "p", ".", "save binary to `path` and add execute permission to it")
rootCmd.PersistentFlags().BoolP("filter-off", "F", false, "turn off auto-filtering feature")
}
// parse user/repo[#tagname] arg
func parseArg(repoPath string) (repo string, tag string) {
seperateTag := strings.Split(repoPath, "#")
if len(seperateTag) == 2 {
tag = seperateTag[1]
}
repo = seperateTag[0]
return repo, tag
}

ghdl/gh.go

Lines 1 to 146 in 11c9c7c

package ghdl
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"path/filepath"
"regexp"
"runtime"
"strings"
"github.com/beetcb/ghdl/helper/sl"
)
const (
OS = runtime.GOOS
ARCH = runtime.GOARCH
)
type GHRelease struct {
RepoPath string
TagName string
}
type APIReleaseResp struct {
Assets []APIReleaseAsset `json:"assets"`
}
type APIReleaseAsset struct {
Name string `json:"name"`
DownloadUrl string `json:"browser_download_url"`
Size int `json:"size"`
}
type AssetNamePredicate func(assetName string) bool
func (gr GHRelease) GetGHReleases(filterOff bool, assetFilter *regexp.Regexp) (*GHReleaseDl, error) {
var tag string
if gr.TagName == "" {
tag = "latest"
} else {
tag = "tags/" + gr.TagName
}
// Os-specific binaryName
binaryName := filepath.Base(gr.RepoPath) + func() string {
if runtime.GOOS == "windows" {
return ".exe"
} else {
return ""
}
}()
apiUrl := fmt.Sprint("https://api.github.com/repos/", gr.RepoPath, "/releases/", tag)
// Get releases info
req, err := http.NewRequest("GET", apiUrl, nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/vnd.github.v3+json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
} else if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("requst to %v failed: %v", apiUrl, resp.Status)
}
defer resp.Body.Close()
byte, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var respJSON APIReleaseResp
if err := json.Unmarshal(byte, &respJSON); err != nil {
return nil, err
}
releaseAssets := respJSON.Assets
if len(releaseAssets) == 0 {
return nil, fmt.Errorf("no binary release found")
}
// Filter or Pick release assets
matchedAssets := func() []APIReleaseAsset {
if filterOff {
return releaseAssets
}
// The common predicate rule suits for both OS and ARCH
osArchPredicate := func(match string) AssetNamePredicate {
return func(assetName string) bool {
if strings.Contains(assetName, match) {
return true
}
if match == "amd64" {
return strings.Contains(assetName, "x64")
}
return strings.Contains(assetName, "x64_64")
}
}
predicates := []AssetNamePredicate{
osArchPredicate(OS),
osArchPredicate(ARCH),
}
if assetFilter != nil {
predicates = append(predicates, func(assetName string) bool {
return assetFilter.MatchString(assetName)
})
}
return filterAssets(releaseAssets, predicates)
}()
matchedIdx := 0
if len(matchedAssets) != 1 {
var choices []string
for _, asset := range matchedAssets {
choices = append(choices, asset.Name)
}
idx := sl.Select(&choices)
matchedIdx = idx
}
asset := matchedAssets[matchedIdx]
return &GHReleaseDl{binaryName, asset.DownloadUrl, int64(asset.Size)}, nil
}
func filterAssets(assets []APIReleaseAsset, predicates []AssetNamePredicate) []APIReleaseAsset {
ret := assets
for _, p := range predicates {
ret = filter(ret, p)
}
return ret
}
// filterAssets assets using the provided predicates, falling back to the default assets if no match is found
func filter(assets []APIReleaseAsset, predicate AssetNamePredicate) []APIReleaseAsset {
var ret []APIReleaseAsset
for _, asset := range assets {
lowerName := strings.ToLower(asset.Name)
if predicate(lowerName) {
ret = append(ret, asset)
}
}
if len(ret) == 0 {
return assets
}
return ret
}

I also found the following external resources that might be helpful:

Summaries of links found in the content:

ghdl/dl.go

Line 72 in f53255a

// Package format `deb` `rpm` `apk` will be downloaded directly
:

The page you provided does not contain any content or code snippets. Therefore, it is not possible to provide a summary of the page relevant to the problem.

Step 2: 🧐 Snippet Analysis

From looking through the relevant snippets, I decided to make the following modifications:

File Path Proposed Changes
dl.go In the DlTo method of the GHReleaseDl struct, replace the panic statement in the starter function with a return of the error. This will allow the error to be propagated up to the caller function.
ghdl/main.go In the Run function of the rootCmd variable, add error handling for the call to the DlTo method. If an error is returned, print an error message and exit the program with a non-zero status code.

Step 3: 📝 Planning

I have created a plan for writing the pull request. I am now working my plan and coding the required changes to address this issue. Here is the planned pull request:

Refactor file download error handling
sweep/refactor-file-download-error-handling

Description

This PR refactors the error handling in the file download process to provide a more robust and flexible error handling mechanism.

Summary of Changes

  • In dl.go:
  • Replaced the panic statement in the DlTo method of the GHReleaseDl struct with an error return. This ensures that errors during the file download process are returned to the caller function instead of abruptly stopping the program execution.
  • In ghdl/main.go:
  • Updated the Run function of the rootCmd variable to handle the error returned by the DlTo method. If an error is returned, an error message is printed and the program exits with a non-zero status code.

These changes improve the code's reliability and maintainability by providing a more controlled and predictable error handling mechanism.

Step 4: ⌨️ Coding

I have finished coding the issue. I am now reviewing it for completeness.

Step 5: 🔁 Code Review

Success! 🚀

To recreate the pull request, leave a comment prefixed with "sweep:" or edit the issue.
Join Our Discord

@kevinlu1248
Copy link

Hey sorry about this. I'm one of the developers of Sweep. We're taking a look at this issue.

@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023

Thank you for the active support, the description and comments of the issue might be a bit vague though, tbh

But I see the generated plan seems doable.

@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023

The plan actually is my solution that has been committed yesterday, maybe that causes conflicts and preventing a PR from been created

@wwzeng1
Copy link

wwzeng1 commented Aug 5, 2023

Yep that would cause an issue

@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023

sweep: figure out a better solution to do 1f17aa9

@beetcb
Copy link
Owner Author

beetcb commented Aug 5, 2023

Yep that would cause an issue

Tricky one here 🤣

@beetcb beetcb changed the title Shall add .exe to directly-download list sweep: figure out a better solution to do https://github.com/beetcb/ghdl/commit/1f17aa96e33c912de7557612df3e628d5f91ebaf Aug 5, 2023
@sweep-ai sweep-ai bot mentioned this issue Aug 5, 2023
Open
@kevinlu1248
Copy link

Ya it's a problem at the execution stage. We have a solution in the works that I'm just testing right now.

@kevinlu1248
Copy link

Hey @beetcb we just deployed the fix. It should do a lot better now at execution.

@beetcb
Copy link
Owner Author

beetcb commented Aug 8, 2023

thanks !

@beetcb beetcb closed this as completed Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
sweep Assigns Sweep to an issue or pull request.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Refactor file download error handling beetcb/ghdl
3 participants
@kevinlu1248 @wwzeng1 @beetcb