Doris 2.0.2 FE Startup failed

[chaoxxin]

Environment: Macbook M1, Docker desktop 4.24.2, enough ram and disk space.

ERROR:

2023-11-12 16:44:49,041 INFO (main|1) [Env.loadImage():1703] image does not exist: /opt/apache-doris/fe/doris-meta/image/image.0
2023-11-12 16:44:59,723 ERROR (main|1) [BDBEnvironment.setup():198] error to open replicated environment. will exit.
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) fe_4f6c3a60_47e2_4a00_84c6_d1895341e0c8(-1):/opt/apache-doris/fe/doris-meta/bdb Can not register /opt/apache-doris/fe/doris-meta/bdb or its sub-directories to WatchService. UNEXPECTED_EXCEPTION: Unexpected internal Exception, may have side effects. Environment is invalid and must be closed.

Deployment method 1:

apache/doris:build-env-ldb-toolchain-no-avx2-latest

as the build environment image.

Both tried:

• apache-doris-2.0.2-bin-x64-noavx2.tar.gz
• apache-doris-2.0.2-bin-arm64.tar.gz

FE startup log：

2023-11-12 12:58:49,543 INFO (main|1) [DorisFE.start():141] Doris FE starting...
2023-11-12 12:58:49,631 INFO (main|1) [FrontendOptions.analyzePriorityCidrs():186] configured prior_cidrs value: 172.17.0.2/24
2023-11-12 12:58:49,651 INFO (main|1) [FrontendOptions.initAddrUseIp():99] local address: /172.17.0.2.
2023-11-12 12:58:50,706 INFO (main|1) [ConsistencyChecker.initWorkTime():106] consistency checker will work from 15:00 to 15:00
2023-11-12 12:58:54,893 INFO (main|1) [PrivTable.addEntry():73] add priv entry: Node_priv Admin_priv 
2023-11-12 12:58:54,895 INFO (main|1) [PrivTable.addEntry():73] add priv entry: Admin_priv 
2023-11-12 12:58:55,096 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:information_schema, priv: Select_priv 
2023-11-12 12:58:55,097 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:mysql, priv: Select_priv 
2023-11-12 12:58:55,099 INFO (main|1) [PrivTable.addEntry():73] add priv entry: origWorkloadGroup:normal, priv:Usage_priv 
2023-11-12 12:58:55,100 INFO (main|1) [Auth.createUserInternal():482] finished to create user: 'root'@'%', is replay: true
2023-11-12 12:58:55,101 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:information_schema, priv: Select_priv 
2023-11-12 12:58:55,101 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:mysql, priv: Select_priv 
2023-11-12 12:58:55,102 INFO (main|1) [PrivTable.addEntry():73] add priv entry: origWorkloadGroup:normal, priv:Usage_priv 
2023-11-12 12:58:55,102 INFO (main|1) [Auth.createUserInternal():482] finished to create user: 'admin'@'%', is replay: true
2023-11-12 12:58:55,683 INFO (main|1) [Env.getSelfHostPort():1216] get self node: HostInfo{host='172.17.0.2', port=9010}
2023-11-12 12:58:55,683 INFO (main|1) [Env.getHelperNodes():1270] get helper nodes: [HostInfo{host='172.17.0.2', port=9010}]
2023-11-12 12:58:55,713 INFO (main|1) [Env.getClusterIdAndRole():1141] finished to get cluster id: 698307200, isElectable: true, role: FOLLOWER and node name: fe_3b89d97e_e224_45da_a34f_bd76f69d25dc
2023-11-12 12:58:55,745 INFO (main|1) [Env.loadImage():1703] image does not exist: /opt/doris/doris-meta/image/image.0
2023-11-12 12:59:06,380 ERROR (main|1) [BDBEnvironment.setup():198] error to open replicated environment. will exit.
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) fe_3b89d97e_e224_45da_a34f_bd76f69d25dc(-1):/opt/doris/doris-meta/bdb Can not register /opt/doris/doris-meta/bdb or its sub-directories to WatchService. UNEXPECTED_EXCEPTION: Unexpected internal Exception, may have side effects. Environment is invalid and must be closed.
        at com.sleepycat.je.log.FileDeletionDetector.<init>(FileDeletionDetector.java:145) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.log.FileManager.<init>(FileManager.java:456) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.<init>(EnvironmentImpl.java:646) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.RepImpl.<init>(RepImpl.java:328) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_382]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_382]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_382]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_382]
        at com.sleepycat.je.dbi.DbEnvPool.loadRepImpl(DbEnvPool.java:261) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:195) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:278) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.<init>(Environment.java:258) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:605) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:464) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:538) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at org.apache.doris.journal.bdbje.BDBEnvironment.setup(BDBEnvironment.java:169) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.journal.bdbje.BDBJEJournal.open(BDBJEJournal.java:332) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.persist.EditLog.open(EditLog.java:1121) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.initialize(Env.java:889) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.DorisFE.start(DorisFE.java:163) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.DorisFE.main(DorisFE.java:83) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: java.io.IOException: Function not implemented
        at sun.nio.fs.LinuxWatchService.<init>(LinuxWatchService.java:64) ~[?:1.8.0_382]
        at sun.nio.fs.LinuxFileSystem.newWatchService(LinuxFileSystem.java:47) ~[?:1.8.0_382]
        at com.sleepycat.je.log.FileDeletionDetector.<init>(FileDeletionDetector.java:85) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        ... 20 more

Deployment method 2:

Split apache-doris-2.0.2-bin-x64-noavx2.tar.gz into：

• fe.tar.gz
• be.tar.gz

Dockerfile(FE)

FROM openjdk:8u342-jdk

ENV JAVA_HOME="/usr/local/openjdk-8/" \
    PATH="/opt/apache-doris/fe/bin:$PATH"

ADD ./resource/fe.tar.gz /opt/

RUN apt-get update && \
    apt-get install -y default-mysql-client && \
    apt-get clean && \
    mkdir /opt/apache-doris && \
    cd /opt && \
    mv /opt/fe /opt/apache-doris/fe && \
    rm -rf /opt/fe.tar.gz

ADD ./resource/init_fe.sh /opt/apache-doris/fe/bin
RUN chmod 755 /opt/apache-doris/fe/bin/init_fe.sh
ENTRYPOINT ["/opt/apache-doris/fe/bin/init_fe.sh"]

The content of init_fe.sh is content

Dockerfile(BE)

FROM openjdk:8u342-jdk
ENV JAVA_HOME="/usr/local/openjdk-8/" \
    PATH="/opt/apache-doris/be/bin:$PATH"

ADD ./resource/be.tar.gz /opt/
RUN apt-get update && \
    apt-get install -y default-mysql-client && \
    apt-get clean && \
    mkdir /opt/apache-doris && \
    cd /opt && \
    mv /opt/be /opt/apache-doris/be && \
    rm -rf /opt/be.tar.gz

ADD ./resource/init_be.sh /opt/apache-doris/be/bin
RUN chmod 755 /opt/apache-doris/be/bin/init_be.sh
ENTRYPOINT ["/opt/apache-doris/be/bin/init_be.sh"]

The content of init_be.sh is content

docker-compose.yml

version: "1"
services:
  fe:
    image: doris_fe
    hostname: doris_fe
    container_name: "doris_fe"
    ports:
      - 8030:8030
      - 9030:9030
    environment:
      - FE_SERVERS=fe1:172.18.0.2:9010
      - FE_ID=1
    volumes:
      - /Users/dajiu/Documents/doris_data/fe_data/doris-meta:/opt/apache-doris/fe/doris-meta
      - /Users/dajiu/Documents/doris_data/fe_data/log:/opt/apache-doris/fe/log
    networks:
      doris_net:
        ipv4_address: 172.18.0.2
  be:
    image: doris_be
    hostname: doris_be
    container_name: "doris_be"
    ports:
      - 8040:8040
    environment:
      - MASTER_FE_IP=172.18.0.2
      - CURRENT_BE_IP=172.18.0.3
      - CURRENT_BE_PORT=9050
      - FE_SERVERS=fe1:172.18.0.2:9010
      - BE_ADDR=172.18.0.2:9050
      - PRIORITY_NETWORKS=172.18.0.0/24
    volumes:
      - /Users/dajiu/Documents/doris_data/be_data/storage:/opt/apache-doris/be/storage
      - /Users/dajiu/Documents/doris_data/be_data/log:/opt/apache-doris/be/log
      - /Users/dajiu/Documents/doris_data/be_data/script:/docker-entrypoint-initdb.d
    depends_on:
      - fe  
    networks:
      doris_net:
        ipv4_address: 172.18.0.3

networks:
  doris_net:
    ipam:
      config:
      - subnet: "172.18.0.0/24"

docker-compose -f docker-compose.yml up

(base) dajiu@DajiudeMBP docker % docker-compose -f docker-compose.yml up
[+] Building 0.0s (0/0)                                                                                                                  docker:desktop-linux
[+] Running 3/0
 ✔ Network docker_doris_net                                                                                                                          Created0.0s 
[+] Running 5/4ris_fe                                                                                                                                Created0. ✔ Network docker_doris_net                                                                                                                          Created0.0s fe The requested image's pla0s fe The requested image's pla0s fe The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested     0.0s 
 ✔ Container doris_fe                                                                                                                                Created0.0s  
 ! fe The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested     0.0s 
 ✔ Container doris_be                                                                                                                                Created0.0s 
 ! be The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested     0.0s 
Attaching to doris_be, doris_fe
doris_fe  | 2023-11-12T16:44:32+00:00 [Warn] [Entrypoint]: FE_SERVERS fe1:172.18.0.2:9010
doris_fe  | 2023-11-12T16:44:32+00:00 [Warn] [Entrypoint]: FE_ID 1
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: add priority_networks 172.18.0.0/24 to /opt/apache-doris/be/conf/be.conf
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: this node role is mix
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: ============= init args ================
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: MASTER_FE_IP  172.18.0.2
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: FE_IP_ARRAY = 172.18.0.2
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: CURRENT_BE_IP  172.18.0.3
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: FE_EDIT_LOG_PORT_ARRAY = 9010
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: CURRENT_BE_PORT  9050
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: MASTER_FE = 172.18.0.2:9010
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: RUN_TYPE 
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: CURRENT_FE = 172.18.0.2:9010
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: PRIORITY_NETWORKS  172.18.0.0/24
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: PRIORITY_NETWORKS = 172.18.0.0/24
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: Current FE is Master FE!  No need check fe status！
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: Current FE is Master FE!  No need to register again！
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: Current FE is Master FE!  No need check fe status！
doris_be  | 2023-11-12T16:44:32+00:00 [Warn] [Entrypoint]: start check be register status~
doris_fe  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: Ready to start CURRENT_FE！
doris_be  | 2023-11-12T16:44:32+00:00 [Note] [Entrypoint]: register is failed, wait next~
doris_be  | 2023-11-12T16:44:54+00:00 [Warn] [Entrypoint]: start check be register status~
doris_be  | 2023-11-12T16:44:54+00:00 [Note] [Entrypoint]: register is failed, wait next~
doris_fe exited with code 0

fe.log

2023-11-12 16:44:42,875 INFO (main|1) [DorisFE.start():141] Doris FE starting...
2023-11-12 16:44:42,941 INFO (main|1) [FrontendOptions.initAddrUseIp():99] local address: /172.18.0.2.
2023-11-12 16:44:43,866 INFO (main|1) [ConsistencyChecker.initWorkTime():106] consistency checker will work from 15:00 to 15:00
2023-11-12 16:44:48,161 INFO (main|1) [PrivTable.addEntry():73] add priv entry: Node_priv Admin_priv 
2023-11-12 16:44:48,163 INFO (main|1) [PrivTable.addEntry():73] add priv entry: Admin_priv 
2023-11-12 16:44:48,362 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:information_schema, priv: Select_priv 
2023-11-12 16:44:48,364 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:mysql, priv: Select_priv 
2023-11-12 16:44:48,366 INFO (main|1) [PrivTable.addEntry():73] add priv entry: origWorkloadGroup:normal, priv:Usage_priv 
2023-11-12 16:44:48,368 INFO (main|1) [Auth.createUserInternal():482] finished to create user: 'root'@'%', is replay: true
2023-11-12 16:44:48,369 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:information_schema, priv: Select_priv 
2023-11-12 16:44:48,370 INFO (main|1) [PrivTable.addEntry():73] add priv entry: database privilege.ctl: internal, db: default_cluster:mysql, priv: Select_priv 
2023-11-12 16:44:48,371 INFO (main|1) [PrivTable.addEntry():73] add priv entry: origWorkloadGroup:normal, priv:Usage_priv 
2023-11-12 16:44:48,372 INFO (main|1) [Auth.createUserInternal():482] finished to create user: 'admin'@'%', is replay: true
2023-11-12 16:44:48,970 INFO (main|1) [Env.getSelfHostPort():1216] get self node: HostInfo{host='172.18.0.2', port=9010}
2023-11-12 16:44:48,971 INFO (main|1) [Env.getHelperNodes():1270] get helper nodes: [HostInfo{host='172.18.0.2', port=9010}]
2023-11-12 16:44:49,007 INFO (main|1) [Env.getClusterIdAndRole():1141] finished to get cluster id: 1479190770, isElectable: true, role: FOLLOWER and node name: fe_4f6c3a60_47e2_4a00_84c6_d1895341e0c8
2023-11-12 16:44:49,041 INFO (main|1) [Env.loadImage():1703] image does not exist: /opt/apache-doris/fe/doris-meta/image/image.0
2023-11-12 16:44:59,723 ERROR (main|1) [BDBEnvironment.setup():198] error to open replicated environment. will exit.
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) fe_4f6c3a60_47e2_4a00_84c6_d1895341e0c8(-1):/opt/apache-doris/fe/doris-meta/bdb Can not register /opt/apache-doris/fe/doris-meta/bdb or its sub-directories to WatchService. UNEXPECTED_EXCEPTION: Unexpected internal Exception, may have side effects. Environment is invalid and must be closed.
	at com.sleepycat.je.log.FileDeletionDetector.<init>(FileDeletionDetector.java:145) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.log.FileManager.<init>(FileManager.java:456) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.dbi.EnvironmentImpl.<init>(EnvironmentImpl.java:646) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.rep.impl.RepImpl.<init>(RepImpl.java:328) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_342]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_342]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_342]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_342]
	at com.sleepycat.je.dbi.DbEnvPool.loadRepImpl(DbEnvPool.java:261) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:195) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:278) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.Environment.<init>(Environment.java:258) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:605) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:464) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:538) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	at org.apache.doris.journal.bdbje.BDBEnvironment.setup(BDBEnvironment.java:169) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.journal.bdbje.BDBJEJournal.open(BDBJEJournal.java:332) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.persist.EditLog.open(EditLog.java:1121) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.catalog.Env.initialize(Env.java:889) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.DorisFE.start(DorisFE.java:163) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.DorisFE.main(DorisFE.java:83) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: java.io.IOException: Function not implemented
	at sun.nio.fs.LinuxWatchService.<init>(LinuxWatchService.java:64) ~[?:1.8.0_342]
	at sun.nio.fs.LinuxFileSystem.newWatchService(LinuxFileSystem.java:47) ~[?:1.8.0_342]
	at com.sleepycat.je.log.FileDeletionDetector.<init>(FileDeletionDetector.java:85) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
	... 20 more

[getyou123]

got the same promblem when I try starting a fe process in my docker ,here is my device info: mac m2(16+256) docker(24.0.5)
here is my step:

1. down load the binary file from doris official binary tar
2. start a container with docker pull apache/doris: build-env-ldb-toolchain-latest and docker run -d -it --name: doris \ -p 9030:9030 -p 8030:8030 \ apache/doris:build-env-ldb-toolchain-latest \ /bin/bash
3. copy the binary file into container with docker cp
4. vi fe config file change the ip config
5. start fe with ./bin/start_fe.sh --daemon
6. fail to connect fe & got the same error log like u

can any one fix this ? thx

[chaoxxin]

之前搞复杂了，直接按照这个就可以
https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/zh/docs/get-started/quickstart/mysql-to-doris/