API: Allow Stream to be set in PodLogOptions

[knight42]

What type of PR is this?

/kind feature
/kind api-change

What this PR does / why we need it:

This PR adds the feature gate and API for kubernetes/enhancements#3288 as well as validation rules to prevent inconsistent values. Changes about API, kube-apiserver and kubelet is included in this PR.

Special notes for your reviewer:

I would like to file another PR to update kubectl in case the volume of this PR is too large.

Does this PR introduce a user-facing change?

Add a `Stream` field to `PodLogOptions`, which allows clients to request certain log stream(stdout or stderr) of the container.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/3288-separate-stdout-from-stderr

[knight42]

/priority important-soon
/triage accepted
/cc @SergeyKanzhelev @mrunalp
/assign @liggitt

[k8s-triage-robot]

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

[knight42]

@smarterclayton @SergeyKanzhelev Hi, sorry for the late response I finally managed to rewrite the code a bit, addressing the comments, and I would like to point out that I decided to change the API semantics when Stream is used in conjunction with TailLines.

Right now, when user specifies both Stream and TailLines, the output is the logs from the last TailLines that match the given Stream, rather than the last TailLines logs of the given Stream.

For instance, let's say the logs of a container is as follows:

[stdout] line 1
[stderr] line 2
[stdout] line 3
[stderr] line 4

If a user request the container logs with stream=stderr&tailLines=2, the result is line 4\n, instead of line 2\nline 4\n.

The reason I made this decision is I found out the implementation of the latter requires parsing log entry to extract
the log stream and a buffering mechanism (the memory required is proportional to TailLines) when reading container logs, which is not efficient and prone to DoS attack.

It is worth mentioning that, dockerd takes the same approach, which can be verified by the following steps:

Run an example container:

$ docker run --name testlogs alpine:edge sh -c 'for i in $(seq 1 10); do if [ $(expr $i % 2) -eq 0 ]; then echo stdout $i; else echo stderr $i >&2; fi; done'
stderr 1
stdout 2
stderr 3
stdout 4
stderr 5
stdout 6
stderr 7
stdout 8
stderr 9
stdout 10

Since docker cli does not allow us to specify the stream, we need to call the dockerd API directly. I wrote a small program to return the stdout stream of container logs with user-specified TailLines:

$ cat main.go

package main

import (
	"context"
	"flag"
	"os"

	"github.com/docker/docker/api/types"
	"github.com/docker/docker/client"
	"github.com/docker/docker/pkg/stdcopy"
)

func main() {
	var tail string
	flag.StringVar(&tail, "n", "1", "tail last n lines")
	flag.Parse()

	cli, err := client.NewClientWithOpts(client.WithAPIVersionNegotiation(), client.WithHostFromEnv())
	if err != nil {
		panic(err)
	}
	reader, err := cli.ContainerLogs(context.TODO(), "testlogs", types.ContainerLogsOptions{
		ShowStdout: true,
		Tail:       tail,
	})
	if err != nil {
		panic(err)
	}
	_, err = stdcopy.StdCopy(os.Stdout, os.Stderr, reader)
	if err != nil {
		panic(err)
	}
}

$ ./test -n 2
stdout 10                  #  <---- only the last line that cames from stdout was returned

$ ./test -n 3
stdout 8
stdout 10

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APErebus]

What's the status of this PR? This would be very helpful and it looks like it's stagnant? Is this awaiting the author just for a rebase?

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[knight42]

/reopen

[k8s-ci-robot]

@knight42: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: knight42
Once this PR has been reviewed and has the lgtm label, please ask for approval from smarterclayton. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• api/OWNERS
• pkg/apis/OWNERS
• pkg/features/OWNERS
• pkg/generated/openapi/OWNERS
• pkg/kubelet/server/OWNERS
• pkg/registry/core/pod/OWNERS
• staging/src/k8s.io/api/OWNERS
• staging/src/k8s.io/apiserver/pkg/endpoints/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[knight42]

Since there is too much merge conflict in this PR, I will open a new one.

[knight42]

I have filed a new PR #127360