Error: Failed to stringify code

[Yusufkulcu]

I trade using chatgpt. I am getting the error shown in the screenshot. The file size is large (7mb on average) and the number of lines is high (134881 lines). What is the problem? Can you help me? This is very important to me

[Yusufkulcu]

@jehna please help me

[jehna]

Can you run humanify using --verbose flag and post the output? It gives a lot more debug data to work with

[Yusufkulcu]

Can you run humanify using --verbose flag and post the output? It gives a lot more debug data to work with

I'm trying right now

[Yusufkulcu]

@jehna

[Yusufkulcu]

@jehna

Do you have any suggestions on what I can do to fix the problem?

[jehna]

Hmm. Seems like an issue with some of the dependencies, at least that error does not originate from humanify's code.
In hindsight it seems that using pkgroll was a bad idea, as it seems to mess with the dependencies' stack traces.
I think this would need some extra debugging to solve.

[Yusufkulcu]

Hmm. Bazı bağımlılıklarla ilgili bir sorun gibi görünüyor, en azından bu hata humanify'ın kodundan kaynaklanmıyor. Geriye dönüp bakıldığında pkgroll'u kullanmanın kötü bir fikir olduğu anlaşılıyor, çünkü bağımlılıkların yığın izlerini bozuyor gibi görünüyor. Bunun çözülmesi için biraz daha hata ayıklamaya ihtiyaç olduğunu düşünüyorum.

@jehna

So what can I do as a solution? Solving this code is very important to me

[Yusufkulcu]

How can I send you error details

[jehna]

Whoops, closed this by accident. You could check which part of the bundle throws the error, that would help

[Yusufkulcu]

Whoops, closed this by accident. You could check which part of the bundle throws the error, that would help

@jehna

When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem. I would be grateful if you could help

[Yusufkulcu]

@jehna

Please help me

[Yusufkulcu]

@jehna

Will you help?

[jehna]

@Yusufkulcu thanks for the additional info, but unfortunately I'm not able to respond to bigger work immediately (not being paid for doing ppen source etc)

[jehna]

I think I'll be able to investigate more within a few weeks when I have some time

[Yusufkulcu]

I understood. I would be very happy if I know when the study is concluded. We hope that it will be concluded as soon as possibleI understood. I would be very happy if I know when the study is concluded. Hoping to conclude it as soon as possible 😊

[0xdevalias]

When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem.

@Yusufkulcu I haven't looked too deeply into this, but I suspect the issue is likely not going to be directly within transformFromAstAsync within @babel/core; but more with how it is being used by either humanify's AST parsing code, or how the webcrack project it depends on is using it.

Looking up that "Failed to stringify code" error leads to this source within humanify:

humanify/src/plugins/local-llm-rename/visit-all-identifiers.ts

Lines 57 to 62 in 2ac5d25

	const stringified = await transformFromAstAsync(ast);
	if (!stringified?.code) {
	throw new Error("Failed to stringify code");
	}
	return stringified?.code;
	}

It looks like it throws that when the result from transformFromAstAsync has a falsy value for stringified?.code

Depending on your skill at debugging, it might be useful to see what the value of ast being passed in is, as well as the full value of stringified that is being returned.

Depending on the sensitivity of the code you're running this on, it could also be useful if you're able to attach the code that consistently triggers this area; even better if you're first able to reduce it to a minimal example that triggers the error; as that would make it a lot easier for others to help debug/understand what might be causing it.

[Yusufkulcu]

When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem.

@Yusufkulcu I haven't looked too deeply into this, but I suspect the issue is likely not going to be directly within transformFromAstAsync within @babel/core; but more with how it is being used by either humanify's AST parsing code, or how the webcrack project it depends on is using it.

Looking up that "Failed to stringify code" error leads to this source within humanify:

humanify/src/plugins/local-llm-rename/visit-all-identifiers.ts

Lines 57 to 62 in 2ac5d25

	const stringified = await transformFromAstAsync(ast);
	if (!stringified?.code) {
	throw new Error("Failed to stringify code");
	}
	return stringified?.code;
	}

It looks like it throws that when the result from transformFromAstAsync has a falsy value for stringified?.code

Depending on your skill at debugging, it might be useful to see what the value of ast being passed in is, as well as the full value of stringified that is being returned.

Depending on the sensitivity of the code you're running this on, it could also be useful if you're able to attach the code that consistently triggers this area; even better if you're first able to reduce it to a minimal example that triggers the error; as that would make it a lot easier for others to help debug/understand what might be causing it.

I didn't understand exactly what I was supposed to do:(

[0xdevalias]

I didn't understand exactly what I was supposed to do:(

@Yusufkulcu At a minimum, if you are able to attach the code you're trying to decrypt, that would be really useful to help narrow things down.

[Yusufkulcu]

Tam olarak ne yapmam gerektiğini anlamadım :(

@YusufkulcuEn azından, şifresini çözmeye çalıştığınız kodu ekleyebilirseniz, bu, işleri daraltmanıza yardımcı olmak için gerçekten yararlı olacaktır.

Yes, you are right, I have attached the file below. Github I uploaded it to my Google drive account because it doesn't allow javascript file uploading.

https://drive.google.com/file/d/1H9_g6L32xMchrpi-dIRksz7MyQkmWm2a/view?usp=drivesdk

@jehna
@0xdevalias

[0xdevalias]

I uploaded it to my Google drive account because it doesn't allow javascript file uploading.

@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.

[Yusufkulcu]

Javascript dosya yüklemeye izin vermediği için Google Drive hesabıma yükledim.

@YusufkulcuTeşekkürler. .js dosyasının adını .txt olarak değiştirirseniz genellikle onu doğrudan GitHub'a da yükleyebilirsiniz.

I uploaded it to my Google drive account because it doesn't allow javascript file uploading.

@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.

I uploaded it to my Google drive account because it doesn't allow javascript file uploading.

@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.

Thanks for the info. I reloaded it as txt. I will be waiting for your help. It's an important project for me
main.txt

[0xdevalias]

Thanks for the info. I reloaded it as txt.

@Yusufkulcu Thanks for that. I ran it through the web version of and it seemed to unpack correctly there, which makes it a higher likelihood that the issue is withinhumanify`:

• https://webcrack.netlify.app/

Depending on your use case, I wonder if you might be able to use that unpacked/unmangled version as a starting point for your analysis; even though it doesn't have the 'nice variable renames' that humanify adds on top? You might also be able to run humanify against individual files from that extracted version in the interim as well.

[0xdevalias]

@Yusufkulcu Looking at your screenshot from #111 (comment), it looks like the error is occuring on file 18:

Looking at the code where this happens, it seems to be within unminify, just after webcrack has run to extract the files, and in the main loop where those files are being processed:

humanify/src/unminify.ts

Lines 6 to 30 in c2e0be6

	export async function unminify(
	filename: string,
	outputDir: string,
	plugins: ((code: string) => Promise<string>)[] = []
	) {
	ensureFileExists(filename);
	const bundledCode = await fs.readFile(filename, "utf-8");
	const extractedFiles = await webcrack(bundledCode, outputDir);
	for (let i = 0; i < extractedFiles.length; i++) {
	console.log(`Processing file ${i + 1}/${extractedFiles.length}`);
	const file = extractedFiles[i];
	const code = await fs.readFile(file.path, "utf-8");
	const formattedCode = await plugins.reduce(
	(p, next) => p.then(next),
	Promise.resolve(code)
	);
	verbose.log("Input: ", code);
	verbose.log("Output: ", formattedCode);
	await fs.writeFile(file.path, formattedCode);
	}
	}

Since even with --verbose enabled, humanify doesn't output the filenames it's processing, it makes it a bit harder to debug; but running it through a debugger we can see what 'file 18' of extractedFiles corresponds with anyway (assuming webcrack consistently unpacks the code the same way); which since the output log shows i + 1, that makes it actually entry 17 in the array (because it's 0-indexed); which seems to be the extracted file 1.js:

Looking at the contents of output/1.js, we can see that it's an empty file:

⇒ cat output/1.js | wc -m
       0

As part of the main unminify loop, the code then calls plugins.reduce on the plugins passed in to unminify:

humanify/src/commands/openai.ts

Lines 26 to 30 in c2e0be6

	await unminify(filename, opts.outputDir, [
	babel,
	openaiRename({ apiKey, model: opts.model }),
	prettier
	]);

When run as humanify openai, these plugins are:

• babel (Ref)
• openaiRename({ apiKey, model: opts.model }) (Ref)
• prettier (Ref)

The Failed to stringify code error you're seeing comes from the visitAllIdentifiers function, just after calling babel's transformFromAstAsync:

humanify/src/plugins/local-llm-rename/visit-all-identifiers.ts

Lines 57 to 61 in c2e0be6

	const stringified = await transformFromAstAsync(ast);
	if (!stringified?.code) {
	throw new Error("Failed to stringify code");
	}
	return stringified?.code;

• https://babeljs.io/docs/babel-core#transformfromastasync

Which is called by the openaiRename plugin that is passed to unminify:

humanify/src/plugins/openai/openai-rename.ts

Lines 6 to 38 in c2e0be6

	export function openaiRename({
	apiKey,
	model
	}: {
	apiKey: string;
	model: string;
	}) {
	const client = new OpenAI({ apiKey });
	return async (code: string): Promise<string> => {
	return await visitAllIdentifiers(
	code,
	async (name, surroundingCode) => {
	verbose.log(`Renaming ${name}`);
	verbose.log("Context: ", surroundingCode);
	const response = await client.chat.completions.create(
	toRenamePrompt(name, surroundingCode, model)
	);
	const result = response.choices[0].message?.content;
	if (!result) {
	throw new Error("Failed to rename", { cause: response });
	}
	const renamed = JSON.parse(result).newName;
	verbose.log(`Renamed to ${renamed}`);
	return renamed;
	},
	showPercentage
	);
	};
	}

transformFromAstAsync accepts a parsed ast, and returns an object with code, map, and ast.

Presumably, with an empty file as input, that would result in an empty ast, which when passed to transformFromAstAsync, would result in an empty string returned in the code field.

Given the check that ends up throwing the error you're seeing is looking for a falsy value in stringified.code; the empty string would definitely trigger that:

humanify/src/plugins/local-llm-rename/visit-all-identifiers.ts

Lines 57 to 62 in c2e0be6

	const stringified = await transformFromAstAsync(ast);
	if (!stringified?.code) {
	throw new Error("Failed to stringify code");
	}
	return stringified?.code;
	}

---

@Yusufkulcu @jehna So the above seems to be the debugging as to the 'why' of what is going on; though I'm not familiar enough with the codebase to suggest where the best point of implementing the fix is.

If I was to suggest the most minimal/localised fix, it would be to the if (!stringified?.code) check within visitAllIdentifiers (Ref)

Though an argument could also be made that filtering out empty/similar files at the unminify level could have it's merits (Ref); though that would then make it less generic/flexible, as theoretically a plugin in the chain could make it a non-empty file before the openaiRename actually needs to process it.

So I still think the best fix is probably within visitAllIdentifiers.

In fact, in looking through related issues, this just seems to be a variation of the same root cause that I raised in this issue:

• Better error handling when humanify local ran on empty file (Error: Failed to stringify code) #54

I also think the general CLI output should be improved to explain what is happening better. eg. a message before running webcrack, outputting the actual filenames (not just the file number), etc.

These issues also seem to tangentially relate to some aspect of this latter suggestion:

• Where is the output? #92
• Improve model download speed, progress display, etc #58

[Yusufkulcu]

Thanks for the info. I reloaded it as txt.

@Yusufkulcu Thanks for that. I ran it through the web version of and it seemed to unpack correctly there, which makes it a higher likelihood that the issue is withinhumanify`:

• https://webcrack.netlify.app/

Depending on your use case, I wonder if you might be able to use that unpacked/unmangled version as a starting point for your analysis; even though it doesn't have the 'nice variable renames' that humanify adds on top? You might also be able to run humanify against individual files from that extracted version in the interim as well.

Thanks for the suggested solution. It works largely without "beautifying variable" values. But my goal is to change it completely. It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?

[0xdevalias]

It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?

@Yusufkulcu Do you mean that the process takes a long time in general? Or that having to manually run it against each file takes a long time?

If the latter, you could probably extract them with webcrack, then write a bash script/similar that runs humanify on each of those files individually. I'm not 100% sure how the renames works across module boundaries/for exports/similar.. but I think it already reads/parses each of the split files individually anyway, so I don't think it would be much worse to run it this way..

Actually.. it will probably still be a bit slower because of the startup boilerplate, and then trying to run webcrack against the file each time (even though they are already split)..

While it's not currently a feature, if you're going to need to be re-running the variable renaming process on future versions of the same code, this feature request would probably also be relevant to you:

• More deterministic renames across different versions of the same code #97

---

@jehna It might be a neat/useful feature to be able to run humanify against a source file (or multiple files/directory of files/etc) while bypassing the webcrack step; that way it could be used on files that had already been unpacked/etc with webcrack (or another tool like wakaru) previously.

Edit: I split this last part into its own issue:

• Allow running humanify against one or more source files that have already been unpacked (eg. skip re-running through webcrack) #129

[Yusufkulcu]

It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?

@Yusufkulcu Do you mean that the process takes a long time in general? Or that having to manually run it against each file takes a long time?

If the latter, you could probably extract them with webcrack, then write a bash script/similar that runs humanify on each of those files individually. I'm not 100% sure how the renames works across module boundaries/for exports/similar.. but I think it already reads/parses each of the split files individually anyway, so I don't think it would be much worse to run it this way..

Actually.. it will probably still be a bit slower because of the startup boilerplate, and then trying to run webcrack against the file each time (even though they are already split)..

While it's not currently a feature, if you're going to need to be re-running the variable renaming process on future versions of the same code, this feature request would probably also be relevant to you:

• More deterministic renames across different versions of the same code #97

@jehna It might be a neat/useful feature to be able to run humanify against a source file (or multiple files/directory of files/etc) while bypassing the webcrack step; that way it could be used on files that had already been unpacked/etc with webcrack (or another tool like wakaru) previously.

Edit: Split this last part into its own issue:

• Allow running humanify against one or more source files that have already been unpacked (eg. skip re-running through webcrack) #129

Thank you for your detailed explanation. I'm going to try the parsed source code separately by writing a script that implements the humanify command. I have one more question.

What is the reason why the "Webcrack" package breaks the code into pieces. I divide it into many files in the Outputs folder and I don't know what they are. I wonder if you can explain that. Thank you in advance

[0xdevalias]

Thank you for your detailed explanation. I'm going to try the parsed source code separately by writing a script that implements the humanify command.

@Yusufkulcu It might actually be quicker for you to just clone the humanify repo and temporarily patch the code to skip trying to run against empty files.

I haven't tried it, but a quick/dirty hack fix could just be to add an if check to skip empty files within unminify:

Something like:

  export async function unminify(
    filename: string,
    outputDir: string,
    plugins: ((code: string) => Promise<string>)[] = []
  ) {
    ensureFileExists(filename);
    const bundledCode = await fs.readFile(filename, "utf-8");
    const extractedFiles = await webcrack(bundledCode, outputDir);
  
    for (let i = 0; i < extractedFiles.length; i++) {
      console.log(`Processing file ${i + 1}/${extractedFiles.length}`);
  
      const file = extractedFiles[i];
      const code = await fs.readFile(file.path, "utf-8");
+     if (code) {
      const formattedCode = await plugins.reduce(
        (p, next) => p.then(next),
        Promise.resolve(code)
      );
  
      verbose.log("Input: ", code);
      verbose.log("Output: ", formattedCode);
  
      await fs.writeFile(file.path, formattedCode);
+     }
+     else {
+       verbose.log(`Skipping empty file ${i + 1} (${file.path})`);
+     }
    }
  }

While this wouldn't really be suited for a proper fix, it should hopefully be enough to do what you need to work around this bug in the meantime.

---

What is the reason why the "Webcrack" package breaks the code into pieces. I divide it into many files in the Outputs folder and I don't know what they are. I wonder if you can explain that.

@Yusufkulcu The original source file you provided is a webpack bundle of code (or similar), where the original many code files have been combined into a single file. When webcrack runs on it, it 'undoes' this bundling, and so splits the single file back into many files similar to how it was originally.

[Yusufkulcu]

ensureFileExists(filename);
    const bundledCode = await fs.readFile(filename, "utf-8");
    const extractedFiles = await webcrack(bundledCode, outputDir);
  
    for (let i = 0; i < extractedFiles.length; i++) {
      console.log(`Processing file ${i + 1}/${extractedFiles.length}`);
  
      const file = extractedFiles[i];
      const code = await fs.readFile(file.path, "utf-8");
+     if (code) {
      const formattedCode = await plugins.reduce(
        (p, next) => p.then(next),
        Promise.resolve(code)
      );
  
      verbose.log("Input: ", code);
      verbose.log("Output: ", formattedCode);
  
      await fs.writeFile(file.path, formattedCode);
+     }
+     else {
+       verbose.log(`Skipping empty file ${i + 1} (${file.path})`);
+     }
    }

Thanks for the suggestion, it doesn't give that error anymore, but now I'm getting another error. I get the following error in file 63. I've added the original file and the file where I got the error below. It's a pretty long mistake, so I added some more pictures.

6blF.txt

main.txt

[0xdevalias]

Thanks for the suggestion, it doesn't give that error anymore, but now I'm getting another error. I get the following error in file 63. I've added the original file and the file where I got the error below. It's a pretty long mistake, so I added some more pictures.

6blF.txt

main.txt

..snip..

..snip..

@Yusufkulcu I'm glad that workaround helped get around the first issue :)

SyntaxError: Unexpected token, expected "," (74:70)
72 |      ｝）；
73 |    };
74 |    subscribeHandler.prototype._subscribe = function (_subscribeHandler.syncErrorThrown) {
   |                                                                       ^
75 |      var sourceObject = this.source;
76 |      return sourceObject && sourceObject.subscribe(_subscribeHandler.syncErrorThrown);
77 |    ｝；

That new error you're running into seems to now be the same/similar to the errors being discussed in this issue:

• Syntax Error on Babel? #117

You might like to try the potential solution I described in this comment, RE: cloning the humanify repo and bumping the webcrack version in package.json to the latest version (2.14.1):

• Syntax Error on Babel? #117 (comment)

[0xdevalias]

For context/continuity, @j4k0xb has raised 2 bugfix PR's for the bugs identified in this issue:

• fix: invalid identifiers #133
• fix: empty code error #134