403 error with http-aws-es

[khoa162]

-----------------------My Code--------------------------------
import es from 'elasticsearch';
import awsSdk from 'aws-sdk';
import httpAwsEs from 'http-aws-es';
import AWS from '../../config/aws';

const options = {
host: '....................',
connectionClass: httpAwsEs,
awsConfig: new awsSdk.Config({
credentials: new awsSdk.Credentials(AWS.accessKeyId, AWS.secretAccessKey),
region: AWS.region,
}),
httpOptions: {}
/amazonES: {
region: AWS.region,
accessKey: AWS.accessKeyId,
secretKey: AWS.secretAccessKey
}/
};
const EsClient = new es.Client(options);
-----------------------------------------------ERROR-------------------------------
Trace: { Authorization Exception :: {"path":"/","query":{},"statusCode":403,"response":""}
at respond (D:\Workspace\2017.........\server\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure (D:\Workspace\2017............\server\node_modules\elasticsearch\src\lib\transport.js:266:7)
at IncomingMessage.cleanUp (D:\Workspace\2017....................\server\node_modules\http-aws-es\connector.js:61:9)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
status: 403,
displayName: 'AuthorizationException',
message: 'Authorization Exception',
path: '/',
query: {},
body: undefined,
statusCode: 403,
response: '',
toString: [Function],
toJSON: [Function] }
at D:/Workspace/2017/................./server/app/controllers/EsController.js:46:13
at respond (D:\Workspace\2017.................\server\node_modules\elasticsearch\src\lib\transport.js:326:9)
at checkRespForFailure (D:\Workspace\2017.................\server\node_modules\elasticsearch\src\lib\transport.js:266:7)
at IncomingMessage.cleanUp (D:\Workspace\2017.................\server\node_modules\http-aws-es\connector.js:61:9)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)

I am using http-aws-es and that is the error. It works well when I change the connectionClass from httpAwsEs to 'http'. Could y guys explain to me the reason why I fail to use http-aws-es in this case?

[brewdium]

Hi @khoa162,

I was also getting a 403 error recently with this package, if you want to try this fork where I have changed out the signer quickly to see if that was the problem for you too it might be a quick fix and also help @TheDeveloper track down the source of the issue. Simply add the fork to your package.json as...
"http-aws-es": "https://github.com/NextFaze/http-aws-es"
or fork my fork if you want to, probably the better idea for security reasons if you don't trust me 😅

Hope that helps

[dpiechota]

Hi,

I am also getting 403 error. "User: anonymous is not authorized to perform: es:ESHttpPost on resource: XXX". @spazworm I have tried your for but with no luck.

import { Client } from 'elasticsearch'
import { HttpAmazonESConnector } from 'http-aws-es/connector'
...
let options = {
  hosts: ['https://' + AWSEnvConfig.ElasticsearchDomainClusterDomainEndpoint],
  connectionClass: HttpAmazonESConnector,
  amazonES: { accessKey: iamCredentials.accessKeyId, secretKey: iamCredentials.secretAccessKey, region: iamCredentials.region }
};

let es = Client(options)

[TheDeveloper]

@Mozowski can you check your options object? amazonES property is deprecated in the latest version. see readme for instructions

[dpiechota]

Hi @TheDeveloper , so the code looks like this

import { HttpAmazonESConnector } from 'http-aws-es/connector'
import { AWSEnvConfig } from '../assets/aws-config';
...
let awsConfigNew = new AWS.Config({
  region: iamCredentials.region,
  credentials: new AWS.Credentials(iamCredentials.accessKeyId, iamCredentials.secretAccessKey)
});

console.log(awsConfigNew) // I have nice Config Object here 

let options = {
  hosts: ['https://' + AWSEnvConfig.ElasticsearchDomainClusterDomainEndpoint],
  connectionClass: HttpAmazonESConnector,
  awsConfig: awsConfigNew
};

console.log(options) // Looks good to me {hosts: Array(1), connectionClass: undefined, awsConfig: Config}

let es = Client(options)
console.log(es)

Still have the same 403 (Forbidden) User: anonymous is not authorized to perform: es:ESHttpPost on resource ZZZ. Any help would be appreciated ;)

[TheDeveloper]

@Mozowski thanks. Is that IAM user definitely authorised for es actions? Can you also double check region is being set correctly?

[dpiechota]

@TheDeveloper thanks for quick answer. I have checked credentials I am logging here console.log(awsConfigNew) in Postman and those work perfectly fine. In the Chrome Developer tools I do not see Authorization Header when debugging this 403 Error.

Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:pl-PL,en-US;q=0.9
Connection:keep-alive
Content-Length:0
Host:XXX
Origin:ZZZ
Referer:ZZZ
User-Agent:Mozilla/5.0 (Linux; Android 6.0.1; MI 5s Build/MXB48T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/62.0.3202.84 Mobile Safari/537.36
X-DevTools-Emulate-Network-Conditions-Client-Id
X-Requested-With:YYY

[dpiechota]

@TheDeveloper I have been able to move forward with debugging. It sounds like this is problem with AWS library.
ERROR Error: Uncaught (in promise): TypeError: AWS.NodeHttpClient is not a constructor
this comes from line this.httpClient = new AWS.NodeHttpClient(); in HttpAmazonESConnector constructor.
I am running my app in ionic and I do not see NodeHttpClient as a property of AWS object.
I believe ionic is running in browser, guess this is the issue.

[TheDeveloper]

@Mozowski thanks for the extra info! That would be the problem.

This module only works on Node because it uses AWS.NodeHttpClient, which isn't present on other platforms.

Browser compatibility is possible though. When running on browser this module could use AWS SDK's XHRClient instead to make requests.

[dpiechota]

Thanks @TheDeveloper !
Changing client this.httpClient = new AWS.XHRClient(); causes new ERROR:

Error: Request error, retrying
  POST https://XXX/_search?query= => Cannot read property 'content-encoding' of undefined

I am not sure if the _search?query= part of above URL is correct query to AWS elasticsearch
I am using es object as follows

let es = Client(options)
console.log(es)

es.search({
  "query": {
    "match_all": {}
  }
})

[TheDeveloper]

@Mozowski unfortunately it's not quite a simple hot-swap, we would need to integrate the XHRClient

[dpiechota]

Ok, thank you for support @TheDeveloper !

[TheDeveloper]

@Mozowski I've just pushed 4.0.0 to use AWS XHRClient in browser. Let me know if that works for you

[dpiechota]

@TheDeveloper everything is working fine! Thank you for great support!