Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraper flaky4 all #15

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Scraper flaky4 all #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
4b410eb
Refactor collectd from opensus to pdata
ZenoCC-Peng Jul 30, 2023
71d15c5
Make crosslink
ZenoCC-Peng Jul 30, 2023
0824410
Update tag
ZenoCC-Peng Jul 30, 2023
99b61b3
Update tag
ZenoCC-Peng Jul 31, 2023
7820f45
Merge branch 'main' into collectd-pdata-v3
ZenoCC-Peng Aug 1, 2023
656ba6a
Refactor from opensus to pdata
ZenoCC-Peng Aug 1, 2023
1fe7187
Remove redundant number
ZenoCC-Peng Aug 1, 2023
55897dc
Remove redundant number
ZenoCC-Peng Aug 1, 2023
56e889a
merge previous version
ZenoCC-Peng Aug 7, 2023
8d894e2
Merge branch 'main' of https://github.com/ZenoCC-Peng/opentelemetry-c…
ZenoCC-Peng Aug 8, 2023
b350954
Test
ZenoCC-Peng Aug 8, 2023
3a95311
Test
ZenoCC-Peng Aug 8, 2023
0f5bbe1
Test
ZenoCC-Peng Aug 8, 2023
eb818f1
Test
ZenoCC-Peng Aug 8, 2023
a89078e
Extend seconds
ZenoCC-Peng Aug 8, 2023
887345e
Markdown other test
ZenoCC-Peng Aug 9, 2023
a21d2b2
Move struct testCase to TestCollectDServer
ZenoCC-Peng Aug 9, 2023
acf9be4
Change markdown
ZenoCC-Peng Aug 10, 2023
e99a1ae
Remove space
ZenoCC-Peng Aug 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/auto-update-jmx-metrics-component.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,6 @@ jobs:
--base main)

pull_request_number=${url//*\//}

# see the template for change log entry file at blob/main/.chloggen/TEMPLATE.yaml
cat > .chloggen/add-jmx-metrics-gatherer-$VERSION.yaml << EOF
change_type: enhancement
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/build-and-test-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
name: build-and-test-windows
on:
# maybe modify
push:
branches:
- 'main'
- 'releases/**'
- 'scraper-flaky4-all'
tags:
- 'v[0-9]+.[0-9]+.[0-9]+*'
pull_request:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/e2e-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
path: |
~/go/bin
~/go/pkg/mod
key: go-cache-${{ runner.os }}-${{ hashFiles('**/go.sum') }}
key: go-cache-${{ runner.os }}-${{ hashFiles('**/go.sum') }}
- name: Install dependencies
if: steps.go-cache.outputs.cache-hit != 'true'
run: make -j2 gomoddownload
Expand All @@ -89,7 +89,7 @@ jobs:
cd receiver/k8sclusterreceiver
go test -v --tags=e2e
- name: run k8sattributesprocessor e2e tests
run: |
run: |
cd processor/k8sattributesprocessor
go test -v --tags=e2e

1 change: 0 additions & 1 deletion .github/workflows/ping-codeowners-issues.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ jobs:
if: ${{ github.repository_owner == 'open-telemetry' }}
steps:
- uses: actions/checkout@v3

- name: Run ping-codeowners-issues.sh
run: ./.github/workflows/scripts/ping-codeowners-issues.sh
env:
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/ping-codeowners-on-new-issue.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ jobs:
if: ${{ github.repository_owner == 'open-telemetry' }}
steps:
- uses: actions/checkout@v3

- name: Run ping-codeowners-on-new-issue.sh
run: ./.github/workflows/scripts/ping-codeowners-on-new-issue.sh
env:
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/ping-codeowners-prs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ jobs:
if: ${{ github.actor != 'dependabot[bot]' && github.repository_owner == 'open-telemetry' }}
steps:
- uses: actions/checkout@v3

- name: Run ping-codeowners-prs.sh
run: ./.github/workflows/scripts/ping-codeowners-prs.sh
env:
Expand Down
2 changes: 1 addition & 1 deletion receiver/hostmetricsreceiver/hostmetrics_receiver_test.go
View file
Open in desktop
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Windows environments require a 5 seconds waiting period for polling, a 10-second interval proves inadequate and will result in exceeding the required time.

Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ func TestGatherMetrics_EndToEnd(t *testing.T) {
cancelFn()

const tick = 50 * time.Millisecond
const waitFor = 10 * time.Second
const waitFor = 20 * time.Second
require.Eventuallyf(t, func() bool {
got := sink.AllMetrics()
if len(got) == 0 {
Expand Down
22 changes: 21 additions & 1 deletion receiver/hostmetricsreceiver/internal/scraper/loadscraper/load_scraper.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,11 @@ func (s *scraper) shutdown(ctx context.Context) error {
return stopSampling(ctx)
}

// scrape
func (s *scraper) scrape(ctx context.Context) (pmetric.Metrics, error) {

var sleepTimeSecs = 5
var overTimeMins = 5

if s.skipScrape {
return pmetric.NewMetrics(), nil
}
Expand All @@ -88,10 +91,27 @@ func (s *scraper) scrape(ctx context.Context) (pmetric.Metrics, error) {
ctx = context.WithValue(ctx, common.EnvKey, s.config.EnvMap)

avgLoadValues, err := s.load(ctx)

if err != nil {
return pmetric.NewMetrics(), scrapererror.NewPartialScrapeError(err, metricsLen)
}

// Employing a for loop to load values, as Windows environments may need to wait 5 seconds to acquire data.
startTime := time.Now()
for avgLoadValues.Load1 == 0 && avgLoadValues.Load5 == 0 && avgLoadValues.Load15 == 0 {
avgLoadValues, err = s.load(ctx)
if err != nil {
return pmetric.NewMetrics(), scrapererror.NewPartialScrapeError(err, metricsLen)
}

time.Sleep(time.Duration(sleepTimeSecs) * time.Second)
// If the operation exceeds the allocated time, returns "overtime error."
if time.Since(startTime) > time.Duration(overTimeMins)*time.Minute {
err := errors.New("exceeds time to load data")
return pmetric.NewMetrics(), scrapererror.NewPartialScrapeError(err, metricsLen)
}
}

if s.config.CPUAverage {
divisor := float64(runtime.NumCPU())
avgLoadValues.Load1 /= divisor
Expand Down
6 changes: 0 additions & 6 deletions receiver/hostmetricsreceiver/internal/scraper/loadscraper/load_scraper_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,7 @@ const (
bootTime = 100
)

// Skips test without applying unused rule
var skip = func(t *testing.T, why string) {
t.Skip(why)
}

func TestScrape(t *testing.T) {
skip(t, "Flaky test. See https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10030")
type testCase struct {
name string
bootTimeFunc func(context.Context) (uint64, error)
Expand Down