Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parallel files processing to catp #9

Merged
merged 36 commits into from
Dec 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
c99a12b
WIP parallel
vearutop Dec 20, 2023
ac9ad45
WIP parallel
vearutop Dec 21, 2023
fed3148
WIP parallel
vearutop Dec 21, 2023
15ad396
WIP parallel
vearutop Dec 21, 2023
d84a97d
dddddd
vearutop Dec 21, 2023
99bd59a
dddddd
vearutop Dec 21, 2023
9880f7a
Revert no-progress
vearutop Dec 21, 2023
d0eeb74
Update profiles
vearutop Dec 21, 2023
7ee8679
Use Boyer-Moore strings finder
vearutop Dec 21, 2023
e3756d2
Update PGO
vearutop Dec 21, 2023
e313d1f
Revert "Update PGO"
vearutop Dec 23, 2023
62d258b
Revert "Use Boyer-Moore strings finder"
vearutop Dec 23, 2023
1740557
Fix lint
vearutop Dec 23, 2023
ed4700a
Update default.pgo
vearutop Dec 23, 2023
6a8a907
Try buffered writing
vearutop Dec 23, 2023
fe7a7e6
Add -progress-json flag
vearutop Dec 23, 2023
d234938
Add -progress-json flag
vearutop Dec 23, 2023
18e467b
Add -progress-json flag
vearutop Dec 23, 2023
d014b4f
Add -progress-json flag
vearutop Dec 23, 2023
0deeffa
Add -progress-json flag
vearutop Dec 23, 2023
56b8e31
Add -progress-json flag
vearutop Dec 23, 2023
0731932
Add -progress-json flag
vearutop Dec 23, 2023
a054ee7
Add -progress-json flag
vearutop Dec 23, 2023
96d3ea3
Disable sync for out-dir
vearutop Dec 23, 2023
672b139
Update PGO
vearutop Dec 23, 2023
1fbf23a
Update PGO
vearutop Dec 23, 2023
634cf6c
Update CI
vearutop Dec 24, 2023
8fdf0b0
Update CI
vearutop Dec 24, 2023
211637d
Update CI
vearutop Dec 24, 2023
9fdb599
Update CI
vearutop Dec 24, 2023
6a83dff
Update CI
vearutop Dec 24, 2023
631bce7
Update CI
vearutop Dec 24, 2023
aca220c
Update CI
vearutop Dec 24, 2023
36e97b2
Update CI
vearutop Dec 24, 2023
674858d
Update CI
vearutop Dec 24, 2023
9ccbaa1
Update README
vearutop Dec 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/release-assets.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ on:
- created
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO_VERSION: 1.21.x
GO_VERSION: '1.22.0-rc.1'
LINUX_AMD64_BUILD_OPTIONS: '-tags cgo_zstd'
GOAMD64: v3
jobs:
build:
name: Upload Release Assets
Expand Down
4 changes: 4 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ linters-settings:
check-exported: false
unparam:
check-exported: true
funlen:
lines: 80
cyclop:
max-complexity: 15

linters:
enable-all: true
Expand Down
1 change: 0 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ ifeq ($(DEVGO_PATH),)
endif


export CGO_ENABLED ?= 0
BUILD_PKG = ./cmd/catp
BUILD_LDFLAGS=-s -w

Expand Down
19 changes: 17 additions & 2 deletions cmd/catp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,25 @@ wget https://github.com/bool64/progress/releases/latest/download/linux_amd64.tar
Usage of catp:
-dbg-cpu-prof string
write first 10 seconds of CPU profile to file
-grep string
grep pattern, may contain multiple patterns separated by \|
-dbg-mem-prof string
write heap profile to file after 10 seconds
-grep value
grep pattern, may contain multiple OR patterns separated by \|,
each -grep value is added with AND logic, akin to extra '| grep foo',
for example, you can use '-grep bar\|baz -grep foo' to only keep lines that have (bar OR baz) AND foo
-no-progress
disable progress printing
-out-dir string
output to directory instead of STDOUT
files will be written to out dir with original base names
disables output flag
-output string
output to file instead of STDOUT
-parallel int
number of parallel readers if multiple files are provided
lines from different files will go to output simultaneouslyuse 0 for multi-threaded zst decoder (slightly faster at cost of more CPU) (default 1)
-progress-json string
write current progress to a file
-version
print version and exit
```
Expand Down
Binary file added cmd/catp/bytesContains.pprof
Binary file not shown.
Loading
Loading