[BUG]: 相同的ilogtail 配置，在k8s 里跟宿主机nohup 运行的结果对相同的日志内容处理的结果不一样，如下 #916
-
|
#日志元数据 2023-06-08 11:05:11 | ERROR | org.springframework.kafka.KafkaListenerEndpointContainer#1-0-C-1 | net.people.ai.service.IdentityInfoSpikeProcessor | 133 | Error updating database. Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'The error may exist in URL [jar:file:/Application.jar!/BOOT-INF/classes!/mapper/IdentityInfo2Mapper.xml]The error may involve net.people.ai.entity.mapper.IdentityInfo2Mapper.batchUpdate-InlineThe error occurred while setting parametersSQL: update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ?Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' Error updating database. Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'The error may exist in URL [jar:file:/Application.jar!/BOOT-INF/classes!/mapper/IdentityInfo2Mapper.xml]The error may involve net.people.ai.entity.mapper.IdentityInfo2Mapper.batchUpdate-InlineThe error occurred while setting parametersSQL: update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ?Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' ilogtail 配置k8s 收集输出结果023-06-08 04:05:37 {"time":"2023-06-08T12:05:36.210651561+08:00","source":"stdout","image_name":"harbor.people-ai.net/refactoring/riskcontrol-identity-info-spike:tag_v0.0.2-2","container_name":"riskcontrol-resource-identity-info-spike","pod_name":"riskcontrol-resource-identity-info-spike-65f7bbd98-m2bp9","namespace":"saas","pod_uid":"382b05b9-7b4b-449b-83c4-5699c205e183","container_ip":"10.120.5.119","content":"### Error updating database. Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' ","time":"1686197136"} 非K8s 收集输出结果，显示正确2023-06-08 11:53:49 {"tag:path":"./multiline1.log","logtime":"2023-06-08 11:46:05","level":"ERROR","process":"org.springframework.kafka.KafkaListenerEndpointContainer#1-1-C-1","thread":"net.people.ai.service.IdentityInfoSpikeProcessor","threadid":"133","log":"### Error updating database. Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n### The error may exist in URL [jar:file:/Application.jar!/BOOT-INF/classes!/mapper/IdentityInfo2Mapper.xml]\n### The error may involve net.people.ai.entity.mapper.IdentityInfo2Mapper.batchUpdate-Inline\n### The error occurred while setting parameters\n### SQL: update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ?\n### Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\norg.springframework.jdbc.BadSqlGrammarException: \n### Error updating database. Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n### The error may exist in URL [jar:file:/Application.jar!/BOOT-INF/classes!/mapper/IdentityInfo2Mapper.xml]\n### The error may involve net.people.ai.entity.mapper.IdentityInfo2Mapper.batchUpdate-Inline\n### The error occurred while setting parameters\n### SQL: update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ? , update t_rc_identity_info2 set details = ? where id = ?\n### Cause: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n\tat org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:235)\n\tat org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)\n\tat org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:91)\n\tat org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441)\n\tat com.sun.proxy.$Proxy109.update(Unknown Source)\n\tat org.mybatis.spring.SqlSessionTemplate.update(SqlSessionTemplate.java:288)\n\tat com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:64)\n\tat com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148)\n\tat com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89)\n\tat com.sun.proxy.$Proxy110.batchUpdate(Unknown Source)\n\tat net.people.ai.service.IdentityInfoSpikeProcessor.process2(IdentityInfoSpikeProcessor.java:130)\n\tat net.people.ai.consumer.KafkaConsumer.listenExecutor(KafkaConsumer.java:37)\n\tat sun.reflect.GeneratedMethodAccessor170.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)\n\tat org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)\n\tat org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:48)\n\tat org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:329)\n\tat org.springframework.kafka.listener.adapter.BatchMessagingMessageListenerAdapter.invoke(BatchMessagingMessageListenerAdapter.java:170)\n\tat org.springframework.kafka.listener.adapter.BatchMessagingMessageListenerAdapter.onMessage(BatchMessagingMessageListenerAdapter.java:162)\n\tat org.springframework.kafka.listener.adapter.BatchMessagingMessageListenerAdapter.onMessage(BatchMessagingMessageListenerAdapter.java:58)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeBatchOnMessage(KafkaMessageListenerContainer.java:1601)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchOnMessageWithRecordsOrList(KafkaMessageListenerContainer.java:1592)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchOnMessage(KafkaMessageListenerContainer.java:1550)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeBatchListener(KafkaMessageListenerContainer.java:1485)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchListener(KafkaMessageListenerContainer.java:1378)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1361)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1080)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:988)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' \n update t_rc_identity_info2\n set details = '[{"emails":[' at line 4\n\tat com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)\n\tat com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)\n\tat com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)\n\tat com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)\n\tat com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:370)\n\tat com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)\n\tat com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)\n\tat org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)\n\tat org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)\n\tat sun.reflect.GeneratedMethodAccessor171.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:64)\n\tat com.sun.proxy.$Proxy149.update(Unknown Source)\n\tat org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)\n\tat org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)\n\tat org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)\n\tat sun.reflect.GeneratedMethodAccessor173.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)\n\tat com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:106)\n\tat org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62)\n\tat com.sun.proxy.$Proxy148.update(Unknown Source)\n\tat org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:194)\n\tat sun.reflect.GeneratedMethodAccessor177.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427)\n\t... 29 common frames omitted\n","time":"1686196426"} |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments
-
|
k8s和正常场景下
|
Beta Was this translation helpful? Give feedback.
-
|
service_docker_stdout 这个多行的配置要看下文档：https://ilogtail.gitbook.io/ilogtail-docs/data-pipeline/input/service-docker-stdout#shu-ju-chu-li-can-shu inputs:
这个配置只能采集单行 |
Beta Was this translation helpful? Give feedback.
-
nohup 启动ilogtail 相关的配置#ilogtail_config.json 为空 2023-06-08 13:24:25 [DBG] [global_config.go:64] [func1] host IP: 172.17.243.194, hostname: SLS-Prod-CPU-Private-GC-MQ-01 k8s 部署ilogtail 
|
Beta Was this translation helpful? Give feedback.
-
|
插件日志： 2023-06-08 05:36:29 [WRN] [regex.go:110] [processRegex] [config#/usr/local/ilogtail/./user_yaml_config.d/saas_stdout.yaml,] AlarmType:REGEX_UNMATCHED_ALARM unmatch this log content:2023-06-08 13:36:28.996|resource-author-service-98c75f78-t6d4v:kafkaCustom-RMZK-SAAS-IMAGE-TASK-RESULT-13|INFO|com.rmzk.bus.tools.LogTool - 消息提交成功:{RMZK-SAAS-IMAGE-TASK-RESULT-1=OffsetAndMetadata{offset=749254023, leaderEpoch=4, metadata=''}} logstore: config:config#/usr/local/ilogtail/./user_yaml_config.d/saas_stdout.yaml |
Beta Was this translation helpful? Give feedback.
-
|
问题，解决，主要是在使用source 插件的时候，file 会自动匹配多行文本，但是docker 插件不会匹配多行文本，需要人工配置正则去匹配多行，https://ilogtail.gitbook.io/ilogtail-docs/data-pipeline/input/service-docker-stdout#shu-ju-chu-li-can-shu 看下文档里的示例5 有详细的案例 |
Beta Was this translation helpful? Give feedback.
问题，解决，主要是在使用source 插件的时候，file 会自动匹配多行文本，但是docker 插件不会匹配多行文本，需要人工配置正则去匹配多行，https://ilogtail.gitbook.io/ilogtail-docs/data-pipeline/input/service-docker-stdout#shu-ju-chu-li-can-shu 看下文档里的示例5 有详细的案例