[Question] Marquez with authentication

[TejasRGitHub]

Hi Team,

I am new to Marquez, and even though Marquez can be deployed AWS - https://marquezproject.ai/docs/deployment/running-on-aws,

1. Is there any integrated way in which Marquez UI can be put behind any OIDC based authentication ?
2. Can Marquez be deployed with a firewall or are there configuration which can help doing this ?
3. Is Marquez designed in such a way that it can be used for any enterprise production system or is it just made as a reference to using Open Lineage ?

[boring-cyborg]

Thanks for opening your first issue in the Marquez project! Please be sure to follow the issue template!

[wslulciuc]

Thanks for raising this question, @TejasRGitHub! It's a very common question we get a lot, and we recognize that we need to provide a clearer answer or better guidance. I'll will go through your questions one-by-one with as much detail as I can:

1. Is there any integrated way in which Marquez UI can be put behind any OIDC based authentication?

Not yet! It's been discuss in #1137. We've decided to move forward with shiro (which does support OIDC SSO, for example see Add Single Sign-on to Your Dropwizard Server in 15 Minutes). Auth will be coming in Marquez 0.52.0.

2. Can Marquez be deployed with a firewall or are there configuration which can help doing this?

There's no configuration, but we'd be happy to add any configuration you feel is needed (within reason) to unblock you as we work on introducing shiro in Marquez 0.52.0. Note, we've seen Marquez UI behind a VPN for internal access, and/or Auth0.

3. Is Marquez designed in such a way that it can be used for any enterprise production system or is it just made as a reference to using Open Lineage?

Yes, Marquez does serves as the reference implementation of OpenLineage, but has been deployed as a lineage solution for organizations such as ILUM, Northwestern Mutual, Matillion, Decathlon, and others.

[TejasRGitHub]

Hi @wslulciuc , Thanks a lot for answering my question.

Can you tell me when is Marquez 0.52.0 planned for release ? Can you point me to your roadmap

[wslulciuc]

We've scheduled our 0.52.0 release for 2024-12-18. If you like to see what's coming in 0.51.0 (next upcoming release), see the 0.51.0 milestone (scheduled for 2024-11-24).

[jonathanpmoraes]

Isn't Shiro a JVM based framework? How is this supposed to work with React for the Marquez UI?

[TejasRGitHub]

Hi @wslulciuc @phixMe , could you please tell if the release ( https://github.com/MarquezProject/marquez/milestone/18 ) would still be coming on 2024-12-18 or has it been postponed to a different timeline.