Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Casting literal to the right type and adding alias to select literal #13

Closed
wants to merge 4 commits into from
Closed

Conversation

alvfermar-mp
Copy link
Contributor

@alvfermar-mp alvfermar-mp commented Jul 26, 2022

Solves #12 and #11. I haven't been able to solve #10 as I haven't found a good place to do the replacement, if you give me some guidance, I'll sort that out as well

@ghost ghost self-requested a review July 26, 2022 10:33
@ghost
Copy link

ghost commented Jul 26, 2022

Hey thanks for raising this. I think the numeric type looks good but I am a little confused about the changes related to #11. It looks like the select literal gets aliased to the table name but I am not sure that this is correct. Take the statements below:

-- Alias with AS
WITH example_cte AS (
  SELECT 
      my_alias.first_name, 
       last_name
  FROM `at-data-platform-preprod.training.employee` AS my_alias
)

SELECT *
FROM example_cte;

-- Select literal inserted and current behaviour does not cause failure as it preserves the alias
WITH example_cte AS (
  SELECT 
      my_alias.first_name, 
       last_name
  FROM (SELECT "bob" AS first_name, "bob" AS last_name) AS my_alias
)

SELECT *
FROM example_cte;

-- Alias without AS
WITH example_cte AS (
  SELECT 
      my_alias.first_name, 
       last_name
  FROM `at-data-platform-preprod.training.employee` my_alias
)

SELECT *
FROM example_cte;

-- Select literal inserted and current behaviour does not cause failure as it preserves the alias
WITH example_cte AS (
  SELECT 
      my_alias.first_name, 
       last_name
  FROM (SELECT "bob" AS first_name, "bob" AS last_name) my_alias
)

SELECT *
FROM example_cte

As far as I can tell the current behaviour preserves the bit after the table reference. Are you doing something different in your dbt models? I think if the entire table was explicitly used then the dry runner would not properly alias it like this:


-- Explicitly select from table
WITH example_cte AS (
  SELECT 
      `at-data-platform-preprod.training.employee`.first_name, 
       last_name
  FROM `at-data-platform-preprod.training.employee`
)

SELECT *
FROM example_cte;

-- Need to insert select literal before .first_name as well?
WITH example_cte AS (
  SELECT 
      `at-data-platform-preprod.training.employee`.first_name, 
       last_name
  FROM (SELECT "bob" as first_name, "bob" as last_name)
)

SELECT *
FROM example_cte;

But not sure if this is what you mean in the issue

@alvfermar-mp
Copy link
Contributor Author

alvfermar-mp commented Jul 27, 2022

Hey @connor-charles you are right I hadn't thought about the possibility that the upstream model was already aliased. Maybe we can include a flag to control of the upstream table is aliased or not? I have updated the changes with it. Regarding the entire table reference being explicitly used I don't think that works in BigQuery, see the image attached:
image
Doesn't work if we specify dataset + table either:
image
However in that same case using just the table name works:
image

@ghost
Copy link

ghost commented Jul 27, 2022

Ah ok I see the problem. I think adding the flag will still not work in all cases if the dbt project has mixed styles. Where sometimes we explicitly use the table name in our SELECT and sometimes we don't and sometimes we alias our reference as well. We need a way to correctly insert the select literal for all these cases in the same project.

Would you mind splitting out these PRs so they map one to one to the issues? It would be good to get the numeric types bug out the way so we can focus on a nice solution to this problem. I will move this discussion to issue #11 to talk about all the cases

@alvfermar-mp
Copy link
Contributor Author

alvfermar-mp commented Jul 28, 2022

Perfect, I left a comment on that issue and for now I'll close this PR and will open a new one with the right changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant