Deploy on SCP failed

[pradjais]

Hi Team,
I am getting an error during the MTA deployment to the SCP. The error is something like below:
#Test 1
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /home/jenkins/NEO_HOME/tools/log/console_deploy-mta_1554185390887.log (Permission denied)
#Test 2
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /home/jenkins/NEO_HOME/tools/log/console_deploy-mta_1553752313264.log (Permission denied)

   From what I could understand , when we run the job it tries to create the log file but fails due to permission issue. Please help on this issue.

P.S. /home/jenkins/NEO_HOME/tools/log/ is present in the server.

[marcusholl]

Seems to be related to #16

There are already some pull requests for resolving this issue:

SAP/jenkins-library#632
SAP/jenkins-library#631
SAP/jenkins-library#614

There is a commit combining basically these three commit (... well at the moment with some additional troubleshooting code): This commit is here: marcusholl/fixNeoDeploy on https://github.com/SAP/jenkins-library.

Please try this commit and report if that works for you. The solution we think about right now will be similar.

[pradjais]

Hi Marcus, thanks for the quick response. But I am already using an older commit - 0e7ef41fabce50fcb2c62a16e3bfb3e0c22bdcaa which supports config.properties file. I tried running Piper by replacing config.properties file with config.yml file but it did not work. Can we stick to 0e7ef41fabce50fcb2c62a16e3bfb3e0c22bdcaa commit or do we need to switch to the marcusholl/fixNeoDeploy commit, if so will it support config.properties file.

[pradjais]

Also, in case marcusholl/fixNeoDeploy commit supports only config.yml file, will the Jenkinsfile be the same or we need to make some changes.

[marcusholl]

But I am already using an older commit which supports config.properties file.

Now it is good time for switching to the new config.yml approach, I suppose.

Should not be that hard to do this move:

E.g.:

DEPLOY_HOST=<host.com>
CI_DEPLOY_ACCOUNT=<account>
neoCredentialsId=CI_CREDENTIALS_ID

goes to:

steps:
  neoDeploy:
        credentialsId: CI_CREDENTIALS_ID
        account: <account>
        host: <host.com>

This file should be a good starting point (don't be afraid, your file will be much, much smaller, please provide only the delta, do not repeat the full file ...)

but it did not work

Why?

will the Jenkinsfile be the same

Dont know how your pipeline script (aka Jenkinsfile) does look like.

I would expect something like

@Library('piper-library-os') _

node('xyz') {
    checkout scm 
    setupCommonPipelineEnvironment script: this
    mtaBuild script: this
    neoDeploy: script: this
}

[pradjais]

Hi Marcus , here I have attached the Jenkinsfile and config.yml files. Please have a look at them and let me know if its ok or some changes need to be made.Thanks.
Jenkinsfile.txt
config.yml.txt

[marcusholl]

The Jenkinsfile is ways to complicated. When we started the Jenkinsfile looked basically like this - and the example which you can find here still does (that we should change, by the way ...). But the configuration handling improved a lot in the meantime, so the pipelines could look much more straight-forward.

Basically what you are doing is

mtaBuild
neoDeploy

From having a look on your pipeline I suggest to strip that down to:

node() {
    stage("Clone sources and setup environment"){
        deleteDir()
        checkout scm
        setupCommonPipelineEnvironment script: this
    }
    stage('Build Fiori App') {
        mtaBuild script: this
    }
    stage('Deploy Fiori App') {
        neoDeploy script: this
    }
}

Regarding the config file.
In principle this looks fine. In the meantime we shifted to an approach not to describe build server setup in a project config file (reason: a project might be built on several build servers with different setups, hence it does not make sense to specify pathes to build tools in the project configuration). We expect the build tools to be configured properly (e.g. contained in the PATH, either direcly denoted, or via /usr/local/bin, environment variables set according to the needs of the build tools). In the meantime we rely on docker for the build tools. This means we wrap the build tools in docker images and instanciate docker containers. See: https://hub.docker.com/u/ppiper.

For backward compatibility reasons it is still supported to provide the location of the mta.jar file. Please use

steps:
    mtaBuild:
        mtaJarLocation: <PATH TO JAR>

The neo toolset (if not used via docker (https://hub.docker.com/r/ppiper/neo-cli)) is expected to be configured via PATH//usr/local/bin.

[marcusholl]

By the way: here is the up-to-date documentation: https://sap.github.io/jenkins-library/scenarios/ui5-sap-cp/Readme/

Would also be possible to switch to the step described there. It depends on your needs wrt to including further steps. In case you would like to put more steps around mtaBuild and neoDeploy you have more freedom with a hand-crafted pipeline. If not step fioriOnCloudPlatformPipeline might fit into your needs ...

[marcusholl]

Fixed with #632 in SAP/jenkins-Library. Please switch back to master branch and come back if it does not work. Otherwise close this issue,

[pradjais]

Hi Marcus. Just one query, is this solution stable or we are still working to improve it.
Thanks.

[pradjais]

Also, initially we didn't require Docker images to implement the solution, but now it seems necessary to have Docker images. Please confirm on the same.
Also since the initial Jenkinsfile anf config.properties as mentioned in https://github.com/SAP/jenkins-pipelines has been deprecated, do we have any other repository with the latest Jenkinsfile and config.yml files.
Thanks.

[radsoulbeard]

Also, initially we didn't require Docker images to implement the solution, but now it seems necessary to have Docker images. Please confirm on the same.
Also since the initial Jenkinsfile anf config.properties as mentioned in https://github.com/SAP/jenkins-pipelines has been deprecated, do we have any other repository with the latest Jenkinsfile and config.yml files.
Thanks.

It is not required to use the Docker images but we recommend it.
Here is the up-to-date documentation: https://sap.github.io/jenkins-library/scenarios/ui5-sap-cp/Readme/ with examples how your Jenkins file should look like and also your config.yml file.
Please refer to the Library steps documentation: https://sap.github.io/jenkins-library/
All defaults can be found here https://github.com/SAP/jenkins-library/blob/master/resources/default_pipeline_environment.yml and can be overwritten in your config.yml file.

[pradjais]

Thanks for the response, but let me rephrase myself, as already mentioned "https://github.com/SAP/jenkins-pipelines" repository had prepared Jenkinsfile and config.properties which has now been deprecated. Since we are new to groovy structure we were just wondering if you could provide us with the updated Jenkinsfile and config.properties(now config.yml) files for the same scenario.
Thanks.

[radsoulbeard]

You just need to copy this in your Jenkinsfile:

@Library('piper-lib-os') _

fioriOnCloudPlatformPipeline script:this

And copy this in your configuration file in .pipeline/config.yml

steps:
  mtaBuild:
    buildTarget: 'NEO'
    mtaJarLocation: '<your mtar location>'
  neoDeploy:
    neoHome: '<your NEO sdk location>'
    neo:
      credentialsId: 'NEO_DEPLOY'
      account: 'your-account-id'
      host: 'hana.ondemand.com'

If you want to use the Docker images, just remove mtaJarLocation and neoHome from your config.
This is the same pipeline as you had before for build and deploy. We have hidden the glue code into the steps to clean up the Jenkinsfile.

[pradjais]

We have changed the Jenkinsfile and config.yml files as you suggested but we are getting an error. Please help us with an analysis of the issue. PFA of the console output.
Error Details(2).txt
Thanks.

[pradjais]

Hi Team, can someone please help us analyze the above attached error details.
Thanks.

[radsoulbeard]

Hi, sorry for the late response. We had public holidays for easter.
It seems to be that some plugins are missing. Please install all plugins you can find here: https://github.com/SAP/devops-docker-images/blob/master/jenkins-master/plugins.txt

[pradjais]

Hi, thanks for the reply. Yes some of the plugins were actually missing , the issue has been fixed now . But it seems there is some issue with the deployment to the cloud Neo environment . Please help us with the analysis of the same.
Thanks.
Error Detail.txt
config.yml.txt

[radsoulbeard]

Sorry my fault, I messed up the config. The correct syntax is:

  mtaBuild:
    buildTarget: 'NEO'
    mtaJarLocation: '<your mtar location>'
  neoDeploy:
    neoHome: '<your NEO sdk location>'
    account: 'your-account-id'
    host: 'hana.ondemand.com'
    neo:
      credentialsId: 'NEO_DEPLOY'

Best regards

[pradjais]

Hi there, we have changed the config.yml file as per your suggestion but we are still getting the same issue. Are we missing something?
Thanks.
Error Detail.txt

[radsoulbeard]

Sorry for the inconvenience. I had to try it out and I have a running config.yml now.
Please add your account and try again.
config.yml.txt

[pradjais]

Thanks for the response. We tried with mentioned config.yml file but its still giving the same error, not sure what we are missing. Please have a look into our project and help us analyse the issue.
Thanks.
https://github.com/pratibhakalmath/piper_github.git
Error Detail.txt

[radsoulbeard]

Ok, your .pipeline folder is at the wrong level. You have to move it into the root level next to the Jenkinsfile.

[pradjais]

Corrected the folder structure , thanks for pointing it out. However we are still getting an issue with the deployment. The error details mentions "wrong credentials" which should not be the case . Please help us if we are misiing something here.
Thanks.
Error Detail.txt

[marcusholl]

I see two issues in the log.

The more important issue is:

ERROR: Wrong credentials. Enter correct user name and password. Note that your user will be locked after 5 consecutive failed authentication attempts.

Verify your credentials by emitting e.g. a neo list-mtas (check neo.sh help for details). If you are sure about your credentials maintain these credentials in your Jenkins.

The second thing is:

java.io.FileNotFoundException: /home/jenkins/NEO_HOME/tools/log/session_deploy-mta_1556276560500.log (Permission denied)

I guess /home/jenkins/NEO_HOME is owned by an user which does not allow the user running jenkins to write into that folder structure (e.g. owned by root, but current user is jenkins). I recommand to fix this by adding the user running Jenkins to a group which has write privileges for the folder (and subfolders) mentioned above.

[pradjais]

Hi Marcus, thanks for the response. We are not getting the second issue anymore. But the credentials issue still persists.

Verify your credentials by emitting e.g. a neo list-mtas (check neo.sh help for details). If you are sure about your credentials maintain these credentials in your Jenkins.

Can you please explain the same in more details. Also find the latest error log.
Thanks.
Error Detail.txt

[marcusholl]

We are not getting the second issue anymore.

Congratulations, what did you change?

Can you please explain [...]

You have apparently maintained invalid credentials. Be careful! Using the wrong credentials again and again will end up in a locked account after 5 consecutive calls with a wrong password.

The most easy thing in this case is to emit a call which only reads, but does not write. -> neo list-mtas

Try in a command line (bash) on your Jenkins instance or on any other suitable machine having the neo toolset installed:

neo.sh list-mtas -h YOUR_HOST -u YOUR_USER

With regards to the call: check also the help: neo.sh list-mtas --help.

You will be asked for the password: Password for your user: Provide the password on the prompt.

If your call succeeds you know you have the right credentials. If the call does not successed (e.g. ending with something like

(!) ERROR: Wrong credentials. Enter correct user name and password. Note that your user will be locked after 5 consecutive failed authentication attempts.
If you need help, provide the output of the command and attach the log directory [PATH_TO_LOG]

you know your credentials are really wrong.

In case is works on the command line: check the credentialsId you maintained in .pipeline/config.yml ( it defaults to CI_CREDENTIALS_ID, in case there is nothing configured you use this identifier). The corresponding credentialsId must exist under "Credentials" (... see the navigation bar on the left side of the Jenkins UI) and the username/password maintained there must be the same like you verified with your call above. I believe you have configured the credentials under the right credentials id (otherwise you would get a different error message) but with a wrong username/password.

Maybe there are users not working with user/password authentification (but maybe with a certificate based authentification). Maybe you should also clarify the details in that direction. But wrt authentification modes I'm not an expert.

[pradjais]

Hi Marcus, we tried to execute the commands with the corresponding cloud credentials but failed with an error message similar to the one you mentioned. But these are the same credential detaild we logon with on SCP , how come its failing here , how can we approach on this ? Please guide us.
Thanks.

[marcusholl]

I have no idea how I can help here. The message is clear: we are dealing with wrong credentials.

I would like to suggest to clarify this with some kind of a admin for your cloud account.

I guess it is basically about permissions. Maybe the user is only valid for UI, but not for the endpoint used by the neo toolset, but the message is blurry, but this is only a wild guess.

My problem here is: I'm not familiar with the details here (permission setup in SCP), so I guess I'm the wrong contact for resolving the details with regards to that user/account management story here.

[pratibhakalmath]

Hi Marcus, sorry was on leave so could not respond. The credentials issue has been resolved now but we are still not able to deploy it on SCP. Please help us with the analysis of the same.
Thanks.

[pratibhakalmath]

Error Detail.txt

[marcusholl]

Thanks for the update.

In the log we can read:

ERROR: Response status code '403' and message 'Forbidden'.

This indicates that the user you are using does not have the permission to perform the deployment. I fear this is something I cannot help. I would like to suggest to clarify the details with the admins of your account (ask them to adjust the permission or to provide you with another user having the appropriate permissions).

[pradjais]

Hi Marcus, the issue has been resolved . Thanks for all your support.