Flowchart of how this works

[neoOpus]

Hi,

Could you provide in the readme a graph or flowchart illustrating how this works, or generate one using LLM to help me understand the code better? I'm particularly curious about whether multiple passes are made to enhance the final code. I tried with ChatGPT but I don't have the Plus, and it just failed to generate one...

[jehna]

Sorry, what is it that you're trying to do? I'm wondering if you're trying to understand the code to educate/help or if you're trying to use the free ChatGPT to run Humanify?

In case it's the latter, there's unfortunately no way to use without paying to OpenAI. But you don't need the plus subscription, only a pay-as-you-go API access. Humanify uses some clever tricks on API level to force the output to include only the renames, so that's why it would not work with the basic chatgpt.com

[jehna]

If it's the former, I'd love to refactor the code to make it easier to approach!

On pseudocode level humanify works about like this:

const ast = parse(minifiedCode)
for (const varName of ast.identifiers) {
  const newName = aiModel.askToRename(varName, surroundingCode(varName, ast))
  ast.rename(varName, newName)
}
save(ast.toString())

So at least one request to ChatGPT is made per variable. The actual rename is made without ChatGPT. Does this answer your question?

[neoOpus]

No, my mistake for not being clear enough in my message, I wanted mainly to see how this works internally via a graph that explain the steps your tool do to deliver the finally result... a visual representation.

eg.

Input > unminification > code parsing > looking for patterns via (something) > beautification > ESLint > LLM variable replacement > Loop x times > Output

If this still doesn't make sense, forget about this, I don't want to take up your development time.

[neoOpus]

If it's the former, I'd love to refactor the code to make it easier to approach!

On pseudocode level humanify works about like this:

const ast = parse(minifiedCode)
for (const varName of ast.identifiers) {
  const newName = aiModel.askToRename(varName, surroundingCode(varName, ast))
  ast.rename(varName, newName)
}
save(ast.toString())

So at least one request to ChatGPT is made per variable. The actual rename is made without ChatGPT. Does this answer your question?

Yes that would be enough for the moment, I will have to examine the source-code, if I still have questions then I will comeback (I just wanted ChatGPT to generate the graph but I don't have the Plus and tried several times then told me that I reached the limit, I will try again today. As this can give anyone a clear idea of the workflow.

[0xdevalias]

I wanted mainly to see how this works internally via a graph that explain the steps your tool do to deliver the finally result... a visual representation.

I will have to examine the source-code

@neoOpus I sort of looked at this myself the other day to understand things better; I found it easiest to start at the CLI command, which call unminify and provides a pipeline of plugins to apply to it (babel, openaiRename, prettier):

humanify/src/commands/openai.ts

Lines 20 to 30 in a518471

	.action(async (filename, opts) => {
	if (opts.verbose) {
	verbose.enabled = true;
	}
	const apiKey = opts.apiKey ?? env("OPENAI_API_KEY");
	await unminify(filename, opts.outputDir, [
	babel,
	openaiRename({ apiKey, model: opts.model }),
	prettier
	]);

You can see the definition for unminify here:

humanify/src/unminify.ts

Lines 6 to 27 in a518471

	export async function unminify(
	filename: string,
	outputDir: string,
	plugins: ((code: string) => Promise<string>)[] = []
	) {
	ensureFileExists(filename);
	const bundledCode = await fs.readFile(filename, "utf-8");
	const extractedFiles = await webcrack(bundledCode, outputDir);
	for (const file of extractedFiles) {
	const code = await fs.readFile(file.path, "utf-8");
	const formattedCode = await plugins.reduce(
	(p, next) => p.then(next),
	Promise.resolve(code)
	);
	verbose.log("Input: ", code);
	verbose.log("Output: ", formattedCode);
	await fs.writeFile(file.path, formattedCode);
	}
	}

You can see the openaiRename plugin code here:

humanify/src/plugins/openai/openai-rename.ts

Lines 6 to 38 in a518471

	export function openaiRename({
	apiKey,
	model
	}: {
	apiKey: string;
	model: string;
	}) {
	const client = new OpenAI({ apiKey });
	return async (code: string): Promise<string> => {
	return await visitAllIdentifiers(
	code,
	async (name, surroundingCode) => {
	verbose.log(`Renaming ${name}`);
	verbose.log("Context: ", surroundingCode);
	const response = await client.chat.completions.create(
	toRenamePrompt(name, surroundingCode, model)
	);
	const result = response.choices[0].message?.content;
	if (!result) {
	throw new Error("Failed to rename", { cause: response });
	}
	const renamed = JSON.parse(result).newName;
	verbose.log(`Renamed to ${renamed}`);
	return renamed;
	},
	showPercentage
	);
	};
	}

You can similarly follow the code flows for the local and gemini based CLI options.

[neoOpus]

Thank you so much Glenn for your help! I really appreciate!

I have a clearer idea now, but I still have some reading to do, I am thinking about feeding the source-code to an LLM to make the flowchart from it, but I still didn't find that solution yet!

BTW, I've been reading your profile and I really liked it, we have a similar approach about learning and doing things!