Billing loop for Shopify App Review Team

[StefanoDiLegami]

Issue summary

After I have submitted my app to the Shopify App Store, somehow the Shopify App Review Team gets in a billing redirect loop when I have this following code:

app.jsx

export async function loader({ request }) {
  const { session } = await authenticate.admin(request);
  const shop = session.shop.replace(".myshopify.com", "");
  const { billing } = await authenticate.admin(request);
  const isTest = shop === "teststore" 

  await billing.require({
    plans: ["MONTHLY"],
    isTest: isTest,
    onFailure: async () =>
      billing.request({ plan: "MONTHLY", isTest: isTest }),
  })

shopify.server.ts

const shopify = shopifyApp({
  billing: {
    MONTHLY: {
      trialDays: 3,
      amount: 0.99,
      currencyCode: "USD",
      interval: BillingInterval.Every30Days,
    },
  }
  });

On all my different test stores, and paid stores, I don't have any issues. I think it is related to the special Shopify Staff stores.

I also have a screencast from Shopify App Review Team that shows this issue.

• @shopify/shopify-app-remix package and version: 1.0.4
• Node version: v18.15.0
• Operating system: macOS

Expected behavior

After first approved billing, don't redirect again to billing page.

Actual behavior

It always loops again to the billing page, even after approved billing. (Only for Shopify App Review Team stores)

Steps to reproduce the problem

1. Setup billing with provided code in the remix template.
2. Check if it works on Shopify App Review Team stores.

[paulomarg]

Hi, thank you for reporting this. I'll escalate it to the reviews team to see if we can figure out what is going wrong here.

[francisbeaudoin]

Hey @StefanoDiLegami, could you either provide the app name or the api key that was submitted for review?

Thanks

[StefanoDiLegami]

Hey @StefanoDiLegami, could you either provide the app name or the api key that was submitted for review?

Thanks

Sure, here is the Client ID of the app I submitted for review:
fc333fcc46d236cee3bc41f31a2a9ec4

[paulomarg]

I understand this was already investigated and resolved, so I'm closing the issue. Please let me know if that's not the case.

[StefanoDiLegami]

I understand this was already investigated and resolved, so I'm closing the issue. Please let me know if that's not the case.

@paulomarg Not really, I just completed the Review without billing and aferwards enabled billing again because it works on normal stores.

Check this screencast of App Review Team.

https://screenshot.click/18-10-55631-49002.webm

[paulomarg]

I'm reopening the issue so we can dig into it. I tested adding this code to my /app loader function on a fresh app, and it works:

  await billing.require({
    isTest: true,
    plans: ["test"],
    onFailure: async () =>
      await billing.request({ isTest: true, plan: "test" }),
  });

Based on the screencast, it looks like the require call isn't finding the payment once it's created, so it keeps asking for a new subscription to be created.

It's worth pointing out that isTest should be consistent between the require and request calls, since the API might not find a payment if they don't match (for example if we're requiring a real payment but requesting a test one). Make sure you're setting isTest to the same value as when the review was run!

Could you please check whether that might be the case here?

Another option might also be to increase logging if you can reproduce this scenario, by adding this to your shopifyApp config:

logger: {
  level: LogSeverity.Debug,
  httpRequests: true,
}

this would log the exact mutation that's being run, which should give you a better idea of what's going wrong.

[github-actions]

We are closing this issue because we did not hear back regarding additional details we needed to resolve this issue. If the issue persists and you are able to provide the missing clarification we need, feel free to respond and reopen this issue.

We appreciate your understanding as we try to manage our number of open issues.

[is2ei]

@paulomarg @francisbeaudoin

I'm facing the same issue.

My loader is like this:

export async function loader({ request }) {
  const { admin, billing } = await authenticate.admin(request);

  const response = await admin.graphql(
    `#graphql
      query {
        shop {
          plan {
            displayName
            partnerDevelopment
          }
        }
      }
    `
  )
  const shop = await response.json()

  await billing.require({
    plans: [PLAN.BASIC],
    // https://github.com/Shopify/shopify-api-js/blob/main/docs/reference/billing/check.md#istest
    // https://shopify.dev/docs/apps/billing/purchase-adjustments/free-trials.md#set-up-free-testing
    isTest: shop.data.shop.plan.partnerDevelopment,
    onFailure: async () => {
      return billing.request({
        plan: PLAN.BASIC,
        // https://github.com/Shopify/shopify-api-js/blob/main/docs/reference/billing/request.md#istest
        // https://shopify.dev/docs/apps/billing/purchase-adjustments/free-trials.md#set-up-free-testing
        isTest: shop.data.shop.plan.partnerDevelopment
      })
    }
  })

  return json({ apiKey: process.env.SHOPIFY_API_KEY });
}

An app review specialist in Shopify Apps team tells me that an error/loop occurred in her store but the installation works fine in my development stores.

It's worth pointing out that isTest should be consistent between the require and request calls,

I think I configured it consistently but the issue hasn't been resolved.

---

Environment:
@shopify/shopify-app-remix package and version: 1.1.0
Node version: v18.x.x
Operating system: I'm not sure which OS the app review specialist uses.

[is2ei]

It looks Shopify Apps team's store tries to activate subscription charge twice at the same time. Is that normal behavior?

[AlexHooperDev]

Hi @paulomarg - My eng team are also experiencing the same issue - billing working fine across dev stores but Shopify App review team stating that there is a billing loop/error present when they are reviewing. The Client ID of the app is: 5ff5e00fed990e7813e3bee3dcc80104.

Our team have a hunch that it's because the Shopify App Review team are accessing the test payment gateway on what is a "production" app, which is causing a loop somewhere.

Are you able to assist? The review team keep redirecting back to the API docs which we have followed to a T.

[gerardkabre]

@AlexHooperDev @is2ei Did you manage to sort it out? Having the same issue I think the isTest param sent to the billing API must be kept to true even if being production to allow them to review and test without real payments, and change it back to false once listed

[is2ei]

@gerardkabre

Did you manage to sort it out?

No, but I communicated with the review team and published my app successfully.

Having the same issue I think the isTest param sent to the billing API must be kept to true even if being production to allow them to review and test without real payments, and change it back to false once listed

I'm not sure if it is the right way to do that. Is there any documentation that mention it?

[gerardkabre]

@is2ei I'm not sure if it is the right way to do that. Is there any documentation that mention it?

No 😔 saw this also commented in another issue I can't find now and in a Shopify facebook dev group (they all were just guessing too though)

[MathiasGr]

Hey @Shopify team, would it be possible to either prioritize a fix or communicate to your review team?

My app has been stuck for a month in review now because of this issue, this is a terrible experience for developers embracing your guidance and Remix SDK

[paulomarg]

Hey folks, thank you for your feedback, and sorry for the delayed response. The team is looking into the issue, and I've reiterated the issue.

Thank you for your patience while we try to resolve this!

[paulomarg]

I was working with the teams to find a solution, and it seems like @MathiasGr was correct in that it is related to the staff stores used by the app review team - we're looking into a way to automatically handle these cases in the future, if we can.

However, as per our documentation, devs should actually turn off real purchases when submitting their apps for review, which should also prevent this problem from happening. You can re-enable real purchases after the approval comes through. We've also connected with the app review team to flag that if they run into this case in the future.

I apologize for not mentioning this before, but I hadn't made the connection between staff stores and how they affect billing as the root cause here.

[tommypepsi]

@paulomarg I think this should be reopened. It just happened to us in production on a shopify plus store. There was "sandbox" in the myshopify url name, but it was an actual store. I believe that somehow shopify turned something on for them that forces all recurring charges to be in test mode.

using the query:

{
  shop {
    plan {
      partnerDevelopment
    }
  }
}

partnerDevelopment was false and the plan name was simply "Shopify Plus". I'm guessing shopify review stores have something similar enabled.

What happens is that if we use the logic isTest: shop.data.shop.plan.partnerDevelopment, for these kind of store we are looking for a recurring charge that is not test mode, but then the configuration of the store forces the charge to be test mode. So this causes an infinite loop of trying to find a charge that is not in test mode.

This issue is not really caused by this library, more the fact that shopify lacks any API to recognize these particular stores that have this weird flag that forces the charges to be test mode.

@paulomarg if you have any say in this, I believe that whatever flag is turned on on the review stores or this particular store I got in production, should turn the "partnerDevelopment" variable to true. Or maybe another separate property if this is too complex or doesn't make sense since they are not actual development stores.

For now we added a whitelist in our code since this doesn't seem too common, but this is not really an ideal solution since we'll have to do support every time this happens to a new store.

[paulomarg]

Hey @tommypepsi , thanks for that. I believe you're right - we can't fully trust partnerDevelopment because not all "non-real" stores will have that flag set. I've raised that point with the team, and they're looking into it, but I'll reiterate so that we can improve this flow!

[dani-sanomads]

Do we have any updates on this , @paulomarg ? Still facing this issue for remix app after the billing approval it redirects again.

[MathiasGr]

Yes... this is still a blocking issue 1 year after the initial report... crazy. And now the support team is saying there is not issue anymore while nothing has changed.

[paulomarg]

Hey folks, I'm sorry this is still an issue. We've been working with the teams on this, and we currently have 2 recommendations to work around this issue:

• Use the new Shopify managed pricing, which enables you to define how to charge stores while publishing your app, so you don't have to use the billing API at all to set them up.
  • Currently, the check method won't work without a config for the app, but I have a WIP PR to address that: Allow not filtering by plans in billing checks #1236. So until that's released, you would need to manually write the GraphQL query to look for purchases in a multi-plan scenario.
• Always use test purchases for the app review process - you can update it to use real purchases after the review process