Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

根据日志中的字段指定hdfs上的路径 #45

Open
zhangzhengstrive opened this issue Nov 22, 2016 · 5 comments
Open

根据日志中的字段指定hdfs上的路径 #45

zhangzhengstrive opened this issue Nov 22, 2016 · 5 comments

Comments

@zhangzhengstrive
Copy link

您好，请问怎么根据日志中的某个字段指定hdfs上的路径path
@repeatedly
Copy link
Member

Write an issue in english.

@zhangzhengstrive
Copy link
Author

Hello, How to use the log field to specify the hdfs path

@tagomoris
Copy link
Member

It's impossible right now, but will be supported in future version with Fluentd v0.14.

@maxwellzdm
Copy link

you may use rewrite_tag_filter to rewrite the tag, and load hdfs path from tag

@zhangzhengstrive
Copy link
Author

thank you, let me try.
How analytical chunk? how string into chunk?
chunk.msgpack_each do |tag, time, record| my_domain = record['domain'] end def format(tag, time, record) [tag, time, record].to_msgpack end
But the data stored in hdfs is garbled
compress_context(chunk) do |data| send_data(hdfs_path, data) end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@repeatedly @tagomoris @maxwellzdm @zhangzhengstrive