Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can someone please tell us how to make this plugin use an http_proxy server? #166

Open
gule001 opened this issue Feb 5, 2021 · 3 comments

Comments

@gule001
Copy link

gule001 commented Feb 5, 2021

Can someone please tell us how to make this plugin use an http_proxy server? We tried multiple settings and approaches and nothing too. Is there an official guide on how to do this?

@y0uk1
Copy link

y0uk1 commented Feb 15, 2021

I could handle it by using:
output { amazon_es { hosts => [""] region => "" index => "" proxy => "http://dummyurl:8080" } }

@gule001
Copy link
Author

gule001 commented Feb 16, 2021

Tried it:
output {
amazon_es {
hosts => ["xxxxx.eu-central-1.es.amazonaws.com"]
region => "eu-central-1"
index => "mc-%{+YYYY.MM.dd}"
aws_access_key_id => 'xxxx'
aws_secret_access_key => 'xxxx'
proxy => "http://xxxxx:3128"
}
}

[2021-02-16T11:51:03,274][WARN ][logstash.outputs.amazonelasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://xxxx.eu-central-1.es.amazonaws.com:443/", :error_type=>LogStash::Outputs::Ama
zonElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://xxx.eu-central-1.es.amazonaws.com:443/][Manticore::ConnectTimeout] Connect timed out"}

I know host is reachable via proxy when I use the curl command and this trick:
https://www.cyberciti.biz/faq/linux-unix-curl-command-with-proxy-username-password-http-options/

We are currently running logstash 7.8.0

Plugin version seems to be this:
./vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0.1-java

ggcolrun@degtluv0023:/DBA/logstash/7.8.0> sed -n '140,144p;145q' ./vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0.1-java/lib/logstash/outputs/amazon_es.rb
# HTTP Path at which the Elasticsearch server lives. Use this if you must run Elasticsearch behind a proxy that remaps
# the root path for the Elasticsearch HTTP API lives.
# Note that if you use paths as components of URLs in the 'hosts' field you may
# not also set this field. That will raise an error at startup
config :path, :validate => :string

@gule001
Copy link
Author

gule001 commented Jun 9, 2021

Looks like we still had the SOCKS proxy settings enabled and the two settings were conflicting with each other (Socks proxy setting was not working at all since we had an HTTP proxy not a SOCKS proxy). We had to remove the following change:

In the file:
/etc/logstash/jvm.properties

We had made a modification to the line LS_JAVA_OPTS="…”

We had add the following string in there:

“ -Dhttp.proxyHost=blabla.com -DsocksProxyHost=sme-out-proxy.server.arvato-systems.de -DsocksProxyPort=3128 "

This was resulting in
-DsocksProxyHost=sme-out-proxy.server.arvato-systems.de -DsocksProxyPort=3128

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants