Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lockaheaad methods #34

Open
snussik opened this issue May 12, 2024 · 9 comments
Open

Lockaheaad methods #34

snussik opened this issue May 12, 2024 · 9 comments

Comments

@snussik
Copy link

snussik commented May 12, 2024

If using .shift(-X) and Series[+X], your inicaators take into account future candles data.

@joshyattridge
Copy link
Owner

Hello,

Can you please provide more information on this?
Do you believe this is an error within my code as I don't think this is possible. But please provide me an example if I'm wrong.

Thank you,
Josh

@snussik
Copy link
Author

snussik commented May 12, 2024

@joshyattridge I've noticed, that you are using "future" shifts inside indicators, instead shifting shapes while plotting. For ex. when calculating fvgs. .shift(-2) is looking 2 candles ahead. It's not critical for demonstration, but'll fail during live usage (you'll not have these "next" two candles, before they happen). Smth. about that is written here.

@joshyattridge
Copy link
Owner

@snussik, Thank you for your input, although you should only provide the candles you want the indicator to show. If you don't pass future candles then my indicators can't have a look ahead bias.
During live tests you should pass all of the OHLC candles data, on each new candle, to the FVG function and the FVG function won't return the most recent FVG until all the necessary candles are present within the data to calculate it.

@snussik
Copy link
Author

snussik commented May 13, 2024

@joshyattridge I think that you are wrong with that: as indicator always needs one future candle, during live plotting it'll never have it to make calculations. So you'll never see any data on the chart. If only you shift live data by that one or two candles to the left (.rolling()). But that means, that you are always left behind market on these candles.

@Asite06
Copy link

Asite06 commented May 30, 2024

hi @snussik @joshyattridge. According to freqtrade lookahead bias detection swing_high_low calculation has lookahead bias issue.

`import numpy as np
import pandas as pd
from freqtrade.strategy import IStrategy, merge_informative_pair
from pandas import DataFrame
import sys
import os
sys.path.append(os.path.abspath("../"))
from smartmoneyconcepts.smc import smc

class lookahead(IStrategy):
timeframe = '1h'
stoploss = -0.99
can_short = True

def populate_indicators(self, df: DataFrame, metadata: dict) -> DataFrame:
    df['HighLow'] = smc.swing_highs_lows(df, swing_length=7)['HighLow']
    return df

def populate_entry_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == -1), 'enter_long'] = 1
    return df

def populate_exit_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == 1), 'exit_long'] = 1
    return df

`
Simply i have tested this code with "freqtrade lookahead-analysis -c config.json -s lookahead --timerange 20240401-20240501" and results are :

2024-05-31 01:54:27,358 - freqtrade.optimize.analysis.lookahead - INFO - => lookahead : bias detected! 2024-05-31 01:54:27,359 - freqtrade.optimize.analysis.lookahead_helpers - INFO - Checking look ahead bias via backtests of lookahead.py took 39 seconds. | filename | strategy | has_bias | total_signals | biased_entry_signals | biased_exit_signals | biased_indicators | |--------------+------------+------------+-----------------+------------------------+-----------------------+---------------------| | lookahead.py | lookahead | True | 20 | 20 | 20 | |

I worked on this but i couldnt get rid of bias. As you know almost all other classmethods depends on swing_high_low calculations. Can you check it and show me the way how to fix. Thanks

@Durden-T
Copy link

Durden-T commented Jun 2, 2024

hi @snussik @joshyattridge. According to freqtrade lookahead bias detection swing_high_low calculation has lookahead bias issue.

`import numpy as np import pandas as pd from freqtrade.strategy import IStrategy, merge_informative_pair from pandas import DataFrame import sys import os sys.path.append(os.path.abspath("../")) from smartmoneyconcepts.smc import smc

class lookahead(IStrategy): timeframe = '1h' stoploss = -0.99 can_short = True

def populate_indicators(self, df: DataFrame, metadata: dict) -> DataFrame:
    df['HighLow'] = smc.swing_highs_lows(df, swing_length=7)['HighLow']
    return df

def populate_entry_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == -1), 'enter_long'] = 1
    return df

def populate_exit_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == 1), 'exit_long'] = 1
    return df

` Simply i have tested this code with "freqtrade lookahead-analysis -c config.json -s lookahead --timerange 20240401-20240501" and results are :

2024-05-31 01:54:27,358 - freqtrade.optimize.analysis.lookahead - INFO - => lookahead : bias detected! 2024-05-31 01:54:27,359 - freqtrade.optimize.analysis.lookahead_helpers - INFO - Checking look ahead bias via backtests of lookahead.py took 39 seconds. | filename | strategy | has_bias | total_signals | biased_entry_signals | biased_exit_signals | biased_indicators | |--------------+------------+------------+-----------------+------------------------+-----------------------+---------------------| | lookahead.py | lookahead | True | 20 | 20 | 20 | |

I worked on this but i couldnt get rid of bias. As you know almost all other classmethods depends on swing_high_low calculations. Can you check it and show me the way how to fix. Thanks

@Asite06 That is expected behavior, just shift it.

        hl = smc.swing_highs_lows(dataframe, window)
        bc = smc.bos_choch(dataframe, hl, True)
        hl = hl.shift(window-1)
        bc = bc.shift(window-1)

@Durden-T
Copy link

Durden-T commented Jun 4, 2024

Th

hi @snussik @joshyattridge. According to freqtrade lookahead bias detection swing_high_low calculation has lookahead bias issue.你好 。根据freqtrade的前瞻偏差检测swing_high_low计算是否存在前瞻偏差问题。
import numpy as np import pandas as pd from freqtrade.strategy import IStrategy, merge_informative_pair from pandas import DataFrame import sys import os sys.path.append(os.path.abspath("../")) from smartmoneyconcepts.smc import smc导入 numpy 作为 np 导入 pandas 作为 pd 从 freqtrade.strategy 导入 IStrategy,merge_informative_pair 从 pandas 导入 DataFrame 导入 sys 导入 o​​s sys.path.append(os.path.abspath("../")) 从 smartmoneyconcepts.smc 导入 smc
class lookahead(IStrategy): timeframe = '1h' stoploss = -0.99 can_short = True类lookahead(IStrategy):时间范围='1h'止损=-0.99 can_short = True

def populate_indicators(self, df: DataFrame, metadata: dict) -> DataFrame:
    df['HighLow'] = smc.swing_highs_lows(df, swing_length=7)['HighLow']
    return df

def populate_entry_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == -1), 'enter_long'] = 1
    return df

def populate_exit_trend(self, df: DataFrame, metadata: dict) -> DataFrame:
    df.loc[(df['HighLow'] == 1), 'exit_long'] = 1
    return df

Simply i have tested this code with "freqtrade lookahead-analysis -c config.json -s lookahead --timerange 20240401-20240501" and results are : 我只是用“freqtrade Lookahead-analysis -c config.json -s Lookahead --timerange 20240401-20240501”测试了这段代码,结果是:
2024-05-31 01:54:27,358 - freqtrade.optimize.analysis.lookahead - INFO - => lookahead : bias detected! 2024-05-31 01:54:27,359 - freqtrade.optimize.analysis.lookahead_helpers - INFO - Checking look ahead bias via backtests of lookahead.py took 39 seconds. | filename | strategy | has_bias | total_signals | biased_entry_signals | biased_exit_signals | biased_indicators | |--------------+------------+------------+-----------------+------------------------+-----------------------+---------------------| | lookahead.py | lookahead | True | 20 | 20 | 20 | |
I worked on this but i couldnt get rid of bias. As you know almost all other classmethods depends on swing_high_low calculations. Can you check it and show me the way how to fix. Thanks我致力于此,但无法摆脱偏见。如您所知,几乎所有其他类方法都取决于 swing_high_low 计算。你能检查一下并告诉我如何修复吗?谢谢

@Asite06 That is expected behavior, just shift it. 这是预期的行为,只需改变它即可。

        hl = smc.swing_highs_lows(dataframe, window)
        bc = smc.bos_choch(dataframe, hl, True)
        hl = hl.shift(window-1)
        bc = bc.shift(window-1)

Sadly my solution is wrong. I don't think there is a way to fix it. Too many lookahead and modify past value issues just in swing_highs_lows function.
Still I believe you do need these behavior.
Thanks for your great work.

@Asite06
Copy link

Asite06 commented Jun 5, 2024

Hi @Durden-T
Actually, I got rid of the bias in swing high-low calculation. I simplified the code and removed consecutive removal because I won't use it on the drawing chart. However, other class methods are very complex, I don't know what to do, but I'm working on it.

@snussik
Copy link
Author

snussik commented Aug 20, 2024

Sadly my solution is wrong. I don't think there is a way to fix it. Too many lookahead and modify past value issues just in swing_highs_lows function. Still I believe you do need these behavior. Thanks for your great work.

@Durden-T I think that the whole code base should be refactored to void lookahead methods. It's not just shifts. Smt. like this:

pip_range = (max(ohlc["high"]) - min(ohlc["low"])) * range_percent

takes into account the whole range.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants