Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MR Process: Prevent many git checkout opperations when unneeded #355

Open
zepumph opened this issue May 15, 2024 · 6 comments
Open

MR Process: Prevent many git checkout opperations when unneeded #355

zepumph opened this issue May 15, 2024 · 6 comments
Assignees

Comments

@zepumph
Copy link
Member

zepumph commented May 15, 2024

This has been biting @jonathanolson during his current maintenance release, for example, here is the current implementation of ReleaseBranch.usesChipper2():

    async usesChipper2() {
      await gitCheckout( this.repo, this.branch );
      const dependencies = await getDependencies( this.repo );
      await gitCheckout( 'chipper', dependencies.chipper.sha );

      const chipperVersion = ChipperVersion.getFromRepository();

      const result = chipperVersion.major !== 0 || chipperVersion.minor !== 0;

      await gitCheckout( this.repo, 'main' );
      await gitCheckout( 'chipper', 'main' );

      return result;
    }

Gross! Why does it need to checkout so many branches?!?!?! Let's try to reduce that so that ReleaseBranches can multitask better.

@zepumph
Copy link
Member Author

zepumph commented May 15, 2024

@jonathanolson and I came to a solution that uses git cat-file blob. This is working really well in testing, and I think it would be a nice improvement for many, many usages of gitCheckout in perennial/js/common.

zepumph added a commit that referenced this issue May 15, 2024
@zepumph
Copy link
Member Author

zepumph commented May 15, 2024

I got to a commit point, but right now I'm just adjusting ReleaseBranch.usesChipper2(). More to come if we don't run into trouble.

Here is how I tested things:

( async () => {

  // TEST ONE
  const ReleaseBranch = require( '../perennial/js/common/ReleaseBranch' );

  const rb12 = new ReleaseBranch( 'acid-base-solutions', '1.2', [ 'phet' ], true );
  const rb13 = new ReleaseBranch( 'acid-base-solutions', '1.3', [ 'phet' ], true );
  console.log( 'acid-base-solutions 1.2 uses chipper 2.0: ', await rb12.usesChipper2() ); // -> false
  console.log( 'acid-base-solutions 1.3 uses chipper 2.0: ', await rb13.usesChipper2() ); // -> true


  /// TEST TWO
  const assert = require( 'assert' );
  const _ = require( 'lodash' );
  const gitCatFile = require( '../perennial/js/common/gitCatFile' );
  const gitCheckout = require( '../perennial/js/common/gitCheckout' );
  const loadJSON = async ( repo, file, branch ) => {
    return JSON.parse( await gitCatFile( repo, file, branch ) );
  };

  // acid base solutions
  // git checkout d1b56e98e01bec64769402037ef60e1e81f90b00
  const testSHA = 'd1b56e98e01bec64769402037ef60e1e81f90b00';
  await gitCheckout( 'acid-base-solutions', testSHA );

  const defaultHEADDependencies = await loadJSON( 'acid-base-solutions', 'dependencies.json' );
  const currentSHADependencies = await loadJSON( 'acid-base-solutions', 'dependencies.json', testSHA );
  const mainDependencies = await loadJSON( 'acid-base-solutions', 'dependencies.json', 'main' );

  assert( _.isEqual( defaultHEADDependencies, currentSHADependencies ) );
  assert( defaultHEADDependencies[ 'acid-base-solutions' ].sha !== mainDependencies[ 'acid-base-solutions' ].sha );
  await gitCheckout( 'acid-base-solutions', 'main' );

  // TEST THREE
  console.log( ( await gitCatFile( 'density-buoyancy-common', 'js/density/model/DensityIntroModel.ts', 'density-1.1' ) ).length, 'DensityIntroModel file chars on density-1.1 branch' );
  console.log( ( await gitCatFile( 'density-buoyancy-common', 'js/density/model/DensityIntroModel.ts' ) ).length, 'DensityIntroModel file chars on main branch' );
} )();

@zepumph
Copy link
Member Author

zepumph commented May 15, 2024

@jonathanolson please report any trouble you find.

@zepumph
Copy link
Member Author

zepumph commented May 16, 2024

@jonathanolson mentioned that we are already doing this with getFileFromBranch So likely this issue is moot, and we can get rid of gitCatFile.

@zepumph zepumph changed the title Prevent many git checkout opperations when unneeded MR Process: Prevent many git checkout opperations when unneeded Oct 2, 2024
@zepumph
Copy link
Member Author

zepumph commented Oct 2, 2024

@samreid and I found another spot for improvement here:

await checkoutTarget( modifiedBranch.repo, modifiedBranch.branch, false );

  • Instead of checking out everything for updateDependencies, I believe we could just look at the modifiedBranch.changedDependencies and only checkout those + the releaseBranch repo.

@zepumph
Copy link
Member Author

zepumph commented Oct 2, 2024

@samreid had a nice thought where we could just ask github directly in certain cases instead of needing to thrash with checkouts and pulls. I don't know if this is totally possible (like to support credentials for private repos etc), but here is a possible way to prevent as many checkouts

// Ensure you're using Node.js v18 or later
// You can check your Node.js version by running `node -v` in your terminal

// Define an asynchronous function to fetch and parse JSON
async function fetchDependencies( sim, branch ) {

  const response = await fetch( `[https://raw.githubusercontent.com/phetsims/${sim}/refs/heads/${branch}/dependencies.json](https://raw.githubusercontent.com/phetsims/$%7Bsim%7D/refs/heads/$%7Bbranch%7D/dependencies.json)` );

  // Check if the response status is OK (status code 200-299)
  if ( !response.ok ) {
    throw new Error( `Network response was not ok. Status: ${response.status} ${response.statusText}` );
  }

  // Parse the response as JSON
  const data = await response.json();
  console.log( data );
}

// Call the function
fetchDependencies( 'density', '1.2' );

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants