Some prompts regularly break the js calling func in reasoner v1

[renato-umeton]

Bug Report

Some prompts regularly break the js calling func in reasoner v1

Steps to Reproduce

Load reasoner v1 and use this prompt: find the first 10 numbers that meet all these criteria: (1) they are part of the fibonacci sequence, (2) they are a prime number plus 7

result: Uncaught exception at line 0: Error: Interrupted
Line not found in code.

Your Environment

• GPT4All version: 3.6.1
• Operating System: macOS 15.2 (M3 Max)
• Chat model used (if applicable): Reasoner v1

[renato-umeton]

[renato-umeton]

Prompt: find the first 10 numbers that meet all these criteria: (1) they are part of the fibonacci sequence, (2) they are a prime number plus 3

Response: To find the first 10 numbers that meet both criteria (being part of the Fibonacci sequence and being a prime number plus 3), we need to follow these steps:

1. Generate Fibonacci numbers.
2. Check if each Fibonacci number is a prime number plus 3.
3. Collect the numbers that satisfy both conditions until we have 10 such numbers.

Let's implement this in JavaScript:

function isPrime(n) {
    if (n <= 1) return false;
    for (let i = 2; i <= Math.sqrt(n); i++) {
        if (n % i === 0) return false;
    }
    return true;
}

function fibonacciGenerator() {
    let a = 0, b = 1;
    while (true) {
        yield a;
        [a, b] = [b, a + b];
    }
}

let count = 0;
const result = [];
const fibGen = fibonacciGenerator();

while (count < 10) {
    const num = fibGen.next().value;
    if (isPrime(num - 3)) {
        result.push(num);
        count++;
    }
}

console.log(result);

Uncaught exception at line 12: SyntaxError: Unexpected token identifier' yield a;```It looks like there was an error in the code. The yield` keyword should be used inside a generator function, and it seems that the generator function is not properly defined.

Let's correct the code:

function isPrime(n) {
    if (n <= 1) return false;
    for (let i = 2; i <= Math.sqrt(n); i++) {
        if (n % i === 0) return false;
    }
    return true;
}

function* fibonacciGenerator() {
    let a = 0, b = 1;
    while (true) {
        yield a;
        [a, b] = [b, a + b];
    }
}

let count = 0;
const result = [];
const fibGen = fibonacciGenerator();

while (count < 10) {
    const num = fibGen.next().value;
    if (isPrime(num - 3)) {
        result.push(num);
        count++;
    }
}

console.log(result);

Uncaught exception at line 0: Error: Interrupted
Line not found in code.```It seems there was an issue with the execution environment. Let's try running the corrected code again to find the first 10 numbers that meet both criteria.

function isPrime(n) {
    if (n <= 1) return false;
    for (let i = 2; i <= Math.sqrt(n); i++) {
        if (n % i === 0) return false;
    }
    return true;
}

function* fibonacciGenerator() {
    let a = 0, b = 1;
    while (true) {
        yield a;
        [a, b] = [b, a + b];
    }
}

let count = 0;
const result = [];
const fibGen = fibonacciGenerator();

while (count < 10) {
    const num = fibGen.next().value;
    if (isPrime(num - 3)) {
        result.push(num);
        count++;
    }
}

console.log(result);

Uncaught exception at line 0: Error: Interrupted
Line not found in code.```

[brankoradovanovic-mcom]

I've tried it yesterday, with the same results. I had a suspicion that the error was somehow related to the use of a generator function, but even without them (e.g. if you specifically say "don't use generator functions"), the error still persists.

[manyoso]

The model in this case is just producing incorrect javascript code and is not correcting itself. Perhaps the error function should give a list of the apis available, and more specific instructions on what the javascript environment looks like?

[manyoso]

Okay, it seems the issue in this particular case is the code is taking too long to run and we abort without properly reporting this to the model. The idea of fleshing out more robust diagnostic information in the event of an error is I think a good one, but this particular case needs a bit more investigation of how to deal with code that takes too long to run either because it is too intensive or because it causes infinite loop or so on.

[manyoso]

This is what happens with llama3.3 70b remote hosted:

NOTE: I specifically increased the timeout to 3 minutes and also included better error information telling the model it needed to write more efficient code.

NOTE 2: ChatGPT also struggles with this, but it has a default timeout that is much longer. The current one we have in production is only ten seconds. Trying to figure out best way to deal with code that might take a very long time to compute. Current best idea is to ask the user if they want the execution to continue after the timeout...

[renato-umeton]

GPT 4o correctly answers the query with two prompts: find the first 10 numbers that meet all these criteria: (1) they are part of the fibonacci sequence, (2) they are a prime number plus 7 followed by write better code. Details and verification.

On the other hand, o1 fails. Details.