Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test OpenStack integration code #36

Open
2 of 5 tasks
altvnk opened this issue Aug 12, 2015 · 7 comments
Open
2 of 5 tasks

Test OpenStack integration code #36

altvnk opened this issue Aug 12, 2015 · 7 comments
Assignees
@altvnk
Labels
blocker sprint-34-2015-Backlog

Comments

@altvnk
Copy link
Contributor

altvnk commented Aug 12, 2015

Need to test OpenStack integration in k8s, at least find out does is able to create LB with external connectivity.

  • Provide cloud.cfg via Ansible and add relevant options to kube-apiserver, kube-scheduler
  • Decide the way how to provide cred's for OpenStack access:
    • read user ENV's with Ansible
    • fill out somewhere in Ansible variables
  • Check if k8s components can reach OpenStack API
  • Check service creation with external loadbalancer option enabled
  • Check OpenStack Cinder persistant volumes plugin
@kenjones-cisco
Copy link
Contributor

When referring to the LB with external connectivity, are you referring to this?
https://github.com/kubernetes/kubernetes/blob/master/docs/user-guide/services.md#type-loadbalancer

@altvnk
Copy link
Contributor Author

altvnk commented Aug 12, 2015

Yes, but with small improvement: https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/openstack/openstack.go

@kenjones-cisco
Copy link
Contributor

On the Openstack Access, I prefer ENVs so within the terraform files I remove all the variables with the openstack provider blocks so everything is picked up from ENV.

But if some have generic users or custom users with limited role then via variables works great. But not sure that will be the most common case.

@kenjones-cisco
Copy link
Contributor

This PR could be of value here as well:
kubernetes/kubernetes#12589

@altvnk
Copy link
Contributor Author

altvnk commented Aug 13, 2015

Started with manual config, but no luck. kube-apiserver fail immediately after start.

Aug 13 11:07:22 k8s-master-01 systemd[1]: Starting Kubernetes API Server...
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: panic: runtime error: invalid memory address or nil pointer dereference
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: [signal 0xb code=0x1 addr=0x0 pc=0x53443d]
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 1 [running]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/cloudprovider/openstack.(*OpenStack).Instances(0x0, 0x0, 0x0, 0x6)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/cloudprovider/openstack/openstack.go:163 +0x8ed
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/cmd/kube-apiserver/app.(*APIServer).Run(0xc820275180, 0xc820190f00, 0x0, 0x10, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:376 +0x1477
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: main.main()
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:48 +0x171
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 5 [chan receive]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: github.com/golang/glog.(*loggingT).flushDaemon(0x1f763e0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x67
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by github.com/golang/glog.init.1
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x297
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 12 [syscall]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: os/signal.loop()
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /usr/lib/golang/src/os/signal/signal_unix.go:22 +0x18
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by os/signal.init.1
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /usr/lib/golang/src/os/signal/signal_unix.go:28 +0x37
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 18 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: github.com/prometheus/client_golang/prometheus.(*registry).Register.func1(0x7f7f9f186990, 0xc8200e1ce0, 0xc8200e40c0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go:216
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by github.com/prometheus/client_golang/prometheus.(*registry).Register
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go:219 +0xb2
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 40 [sleep]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: time.Sleep(0x12a05f200)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /usr/lib/golang/src/runtime/time.go:59 +0xf9
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Until(0x1722340, 0x12a05f200, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:113 +0x61
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Forever(0x1722340, 0x12a05f200)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:93 +0x34
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/util.InitLogs
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/logs.go:50 +0xae
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 41 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Forever(0xc8202b7aa0, 0x3b9aca00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:92
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).Run
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:138 +0x78
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 42 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Forever(0xc8202b7af0, 0x3b9aca00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:92
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).Run
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:138 +0x78
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 43 [chan receive]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: github.com/prometheus/client_golang/prometheus.(*registry).Register(0xc82001c8a0, 0x7f7f9f186990, 0xc8200e1ce0, 0x0, 0x0, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go:229 +0x1c4
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: github.com/prometheus/client_golang/prometheus.Register(0x7f7f9f186990, 0xc8200e1ce0, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go:114 +0x43
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: github.com/prometheus/client_golang/prometheus.MustRegister(0x7f7f9f186990, 0xc8200e1ce0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go:121 +0x2b
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client/metrics.Register.func1()
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/metrics/metrics.go:46 +0x40
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: sync.(*Once).Do(0x1f9f850, 0x1724c90)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /usr/lib/golang/src/sync/once.go:44 +0xe4
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client/metrics.Register()
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/metrics/metrics.go:47 +0x2f
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client.NewRequest(0x7f7f9f1873d0, 0xc8202b43c0, 0x14016a8, 0x3, 0xc8202b2a80, 0x140b020, 0x2, 0x7f7f9f1810e0, 0xc82010ba00, 0x7f7f9f13c050)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/request.go:104 +0x1c
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client.(*RESTClient).Verb(0xc82014d900, 0x14016a8, 0x3, 0xc8200e4060)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/restclient.go:94 +0xb4
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client.(*RESTClient).Get(0xc82014d900, 0xc8200e4060)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/restclient.go:114 +0x36
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client.(*limitRanges).List(0xc8200e8020, 0x7f7f9f13c050, 0xc8200e8040, 0xc8200e4060, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/limit_ranges.go:60 +0x7f
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/plugin/pkg/admission/limitranger.NewLimitRanger.func1(0x0, 0x0, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/plugin/pkg/admission/limitranger/admission.go:96 +0x143
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client/cache.(*ListWatch).List(0xc8202b7b00, 0x0, 0x0, 0x0, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/listwatch.go:63 +0x39
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client/cache.(*Reflector).ListAndWatch(0xc8202b2d80, 0xc82001cc60)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:178 +0xab
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/client/cache.(*Reflector).Run.func1()
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:138 +0x2c
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Until.func1(0xc8202b7b30)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:111 +0x3a
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Until(0xc8202b7b30, 0x3b9aca00, 0x0)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:112 +0x53
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Forever(0xc8202b7b30, 0x3b9aca00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:93 +0x34
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).Run
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:138 +0x78
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 44 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Until(0xc8202cb1c0, 0x3b9aca00, 0xc82016db00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:102
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).RunUntil
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:144 +0x9a
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 45 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Until(0xc8202cb1e0, 0x3b9aca00, 0xc82016db00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:102
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).RunUntil
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:144 +0x9a
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: goroutine 46 [runnable]:
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: k8s.io/kubernetes/pkg/util.Forever(0xc8202b7be0, 0x3b9aca00)
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:92
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: created by k8s.io/kubernetes/pkg/client/cache.(*Reflector).Run
Aug 13 11:07:22 k8s-master-01 kube-apiserver[6878]: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/client/cache/reflector.go:138 +0x78
Aug 13 11:07:22 k8s-master-01 systemd[1]: kube-apiserver.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 13 11:07:22 k8s-master-01 systemd[1]: Failed to start Kubernetes API Server.

@altvnk altvnk self-assigned this Aug 21, 2015
@altvnk
Copy link
Contributor Author

altvnk commented Sep 1, 2015

Finally i get working integration, k8s can login into OpenStack API, trys to create LB and fails.
Continuing my investigation...

@altvnk altvnk added the blocker label Sep 8, 2015
@altvnk
Copy link
Contributor Author

altvnk commented Sep 8, 2015

I'm blocked here because gophercloud Golang library used in OpenStack integration doesn't supports compute API 1.x, only 2.0 and higher. Our OpenStack uses old (deprecated) compute API 1.1.

@altvnk altvnk added ready and removed in progress labels Sep 8, 2015
@peterlamar peterlamar reopened this Sep 8, 2015
@altvnk altvnk removed the ready label Sep 9, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@altvnk altvnk

Labels
blocker sprint-34-2015-Backlog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@altvnk @peterlamar @kenjones-cisco