Remove eval when possible

[denis-migdal]

Eval is very very slow, as JS parser has to be invoked. Moreover, contrary to new Function(), it prevents some optimizations (3x slower).

The list of all eval to check :

• www/src/action_helpers.js: value = b.bytes.$new(b.bytes, eval(value), 'ISO-8859-1')
• www/src/builtin_modules.js: eval(content)
• www/src/builtin_modules.js: eval(content)
• www/src/local_storage_fs.js: for(var attr in $ns){eval('var '+attr+'=$ns["'+attr+'"]')}
• www/src/py2js.js: eval(js)
• www/src/py_builtin_functions.js: eval(f.$content)
• www/src/py_import.js: if(kk[i].charAt(0) == "$"){console.log(eval(kk[i]))}
• www/src/python_parser.js: eval(var ${klass_name} = new $B.ast.${klass_name}())
• www/src/python_parser.js: eval(function_code)
• www/src/python_parser.js: eval('var ' + rule.alias + ' = res')
• www/src/python_parser.js: return eval(rule.action)
• www/src/python_parser.js: eval('var ' + rule.alias + ' = res')
• www/src/python_parser.js: return eval(rule.action)
• www/src/python_parser.js: eval('var ' + rule.items[i].alias + ' = _make')
• www/src/python_parser.js: makes.push(eval(rule.action))
• www/src/python_parser.js: eval('var ' + rule.items[i].alias + ' = _make')
• www/src/python_parser.js: ast = eval(rule.action)
• www/src/libs/hashlib.js: eval(res + "; $B.CryptoJS = CryptoJS;")
• www/src/libs/hashlib.js: eval(res + "; $B.CryptoJS = CryptoJS;")
•  www/src/libs/_jsre.js: for($var in $ns){eval("var " + $var + "=$ns[$var]")}
• www/src/libs/_profile.js: eval($B.InjectBuiltins());
• www/src/libs/profile.js: eval('var $locals'+globals_id+' = {}\nvar $locals_'+locals_id+' = {}')
• www/src/libs/profile.js: eval('$locals' + globals_id + '["' + item1 +
•  www/src/libs/profile.js: eval('$locals' + locals_id + '["' + item[0] + '"] = item[1]')
• www/src/libs/_profile.js: res = eval(js)
• www/src/libs/profile.js: gns = eval('$locals'+globals_id)
• www/src/libs/profile.js: var lns = eval('$locals'+locals_id)
• www/src/libs/_webcomponent.js: var src = eval(code)
• www/src/libs/_webcomponent.js: var webcomp = eval(name) // JS class for component

The list of all new Function to check :

• www/src/ast_to_js.js: return new Function('fct', 'args', generate_args0_str(...args) );
  No ways around it : function is dynamically created, and would requires parsing.
• www/src/builtin_modules.js: f = new Function(js)
• www/src/js_objects.js: var new_js_class = Function('cl_dict', 'bases', body)(cl_dict, bases)
• www/src/loaders.js: var module = new Function(script.js + \nreturn locals)()
• www/src/py_ast.js: ast[kl] = Function(...arg_list, body)
• www/src/py_builtin_functions.js: var exec_func = new Function('$B', 'b',
• www/src/py_complex.js: complex['' + $B.$comps[$op] + ''] = Function('self', 'other',
• www/src/py_float.js: float[__${ops[op]}__] = Function('self', 'other', body)
• www/src/py_float.js: float[__${$B.$comps[op]}__] = Function('self', 'other', body)
• www/src/py_import.js: new Function(module_contents)()
• www/src/py_import.js: var mod = (new Function(module_id, js))(module)
• www/src/py_import.js: var $module = new Function(prefix + parent_id, mod_js)(
• www/src/py_int.js:int.add = Function('self', 'other', op_model)
• www/src/py_int.js:int.mul = Function('self', 'other',
• www/src/py_int.js:int.sub = Function('self', 'other',
• www/src/py_int.js:int.and = Function('self', 'other', model)
• www/src/py_int.js:int.lshift = Function('self', 'other',
• www/src/py_int.js:int.rshift = Function('self', 'other',
• www/src/py_int.js:int.or = Function('self', 'other',
• www/src/py_int.js:int.xor = Function('self', 'other',
• www/src/py_long_int.js:long_int.and = Function('self', 'other', body)
• www/src/py_long_int.js:long_int.or = Function('self', 'other',
• www/src/py_long_int.js:long_int.xor = Function('self', 'other',
• www/src/py_string.js: str[__${comps[op]}__] = Function('_self', 'other',

[PierreQuentel]

The number of evals in the scripts used by the "standard parser" alternative (python_parser.js) made me want to revisit this code and see if there could be a way to make it faster by replacing the eval() by something else.

After some days of hard work, almost all tests of the built-in test suite pass. Unfortunately, even without any eval (except a dozen when the script is loaded), parsing time is still much slower than with the "hand-made" parser in py2js.js...

[denis-migdal]

If you want you can send me an HTML page using the standard parser and I can try to profile it.

It'd be easier to profile if the parsing is launched after clicking on a button, takes 10s+ and prints something when execution is over.

[PierreQuentel]

You can test the standard parser by adding these scripts in the HTML page

<script type="text/javascript" src="/src/string_parser.js"></script>
<script type="text/javascript" src="/src/number_parser.js"></script>
<script type="text/javascript" src="/src/action_helpers.js"></script>
<script type="text/javascript" src="/src/python_parser.js"></script>
<script type="text/javascript" src="/src/full_grammar.js"></script>

and set a __BRYTHON__ attribute in the <body> tag

<body onload="__BRYTHON__.parser_to_ast=1">

All the programs in <script type="text/python">, and the modules they import, will be parsed by the PEG parser in python_parser.js and generate the same AST as the "hand-made" parser (except for a few details that still need fixing, related to PEP 695).

You can compare the parsing time between the 2 implementations, I didn't make performance tests but when running the built-in test suite it is obvious that it's still significantly slower, even without eval().

It's a shame really because it would make the implementation much easier to maintain, and more compliant with standard Python...

[denis-migdal]

Will it also work if I use the __BRYTHON__.runPythonScript("...") (I don't remember the exact name).

I'll try to do a quick test tomorrow.

[austin-schick]

@denis-migdal I did some profiling of the standard parser and wrote about it here: #2354. I thought you might find it interesting!