Enable advanced Istio features by supporting experimental configuration

[strekm]

Description

Istio module is fully managed and supported Istio installation exposing only limited options to configure. Options exposed via Istio CR are fully operational and production ready Istio configurations.

Managed Istio offering is evolving dynamically. To enable early adoption and identification of potential features that can be supported in manage offering it would be nice to provide Istio configuration in experimental offering.

Offered solution should be designed in a way that managed offering is fully separated from experimental offering. There shouldn't be a possibility to apply not fully supported configuration in production ready environment.

Providing this functionality unblocks Istio module users to experiment with configurations or try out alpha / beta functionality that Istio itself provides.

Tasks:

• POC separate experimental / production functionality
• Experimental support for Gateway API
• Experimental image building (including release of manifest as GH release asset)
  - [ ] Gateway API CRDs installation
• Documentation
  - [ ] Blogpost targeting internal teams only
• rollout to experimental channel only

Reasons
unblock istio functionality early adopters

Attachments
will fix: #580

[strekm]

upon receiving feedback from @pbochynski that exposing full configuration is bold move and against our politics to provide managed and production ready solution it was decided that we will be enabling early adopters by providing specific configurations via Istio CRD that will be only available in experimental offering.
Provided solution will give opportunity to test out alpha / beta istio features that later on could be promoted regular offering. For not supported configurations it gives opportunity to collect feedback.
Managing lifecycle of feature, adoption and migration is way easier in this situation. Exposing API in experimental offering prolongs testing time and feedback collection. Migration is easier because of earlier exposed API.
Both solutions: fully configuration, specific configuration have pros and cons. After reviewing them it was decided that we want to expose specific features instead full configuration. We still want to offer opinionated, production ready and fully managed Istio module.

[strekm]

it was decided that it is still too early for this feature to be showcased, even for internal teams.

[strekm]

would be nice to have Gateway API CRDs installed by Kyma Istio module. added #871

[strekm]

removed Gateway API CRDs installation from scope of this task

[strekm]

working on tutorials highlighting exposing httpbin and tcp workload using Gateway API CRs.