Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Remove Maven profile full in favour of using docs for all document generation. #6007

Closed
wants to merge 8 commits into from
Closed

[CI] Remove Maven profile full in favour of using docs for all document generation. #6007

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
75097e0
[CI] Ensure CI-PR executes rest-docs generation as well.
corneil Oct 23, 2024
270fd98
Remove Maven profile full in favour of using docs for both Javadocs a…
corneil Oct 28, 2024
8cf9ece
Up build-docs.sh to provide complete build
corneil Oct 28, 2024
030b89c
Merge branch 'main' into corneil/ensure-ci-pr-tests-documentation-gen…
corneil Oct 28, 2024
410f529
Update surefire in docs profile to use package phase to ensure we can…
corneil Oct 30, 2024
f927c08
Updated tests for new resource versions in log (3.2.1 and 4.0.0) and …
corneil Nov 1, 2024
91b0193
Fix last failing tests.
corneil Nov 2, 2024
d1d71a1
Added plain text test reporter.
corneil Nov 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-snapshot-worker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
./mvnw --help
./spring-cloud-dataflow-package/set-package-version.sh
./mvnw -s .settings.xml package -DskipTests -T 1C --no-transfer-progress
jfrog rt mvn -s .settings.xml install -Pfull,docs -B --no-transfer-progress
jfrog rt mvn -s .settings.xml install -Pdocs -B --no-transfer-progress
jfrog rt mvn -s .settings.xml install -pl spring-cloud-dataflow-package -B --no-transfer-progress
jfrog rt build-publish
PROJECT_VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout)
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/ci-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,15 @@ jobs:
shell: bash
timeout-minutes: 75
run: |
./mvnw -B -s .github/settings.xml -Pdocs clean install --no-transfer-progress
./mvnw -s .github/settings.xml -B -Pdocs clean install --no-transfer-progress -T 1C
- name: Install XML Utils
if: ${{ failure() }}
uses: ./.github/actions/install-xmlutils
- name: Test Errors Report
if: ${{ failure() }}
shell: bash
run: ./src/scripts/print-test-errors.sh

scan:
runs-on: ubuntu-latest
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
export spring_cloud_dataflow_version
./spring-cloud-dataflow-package/set-package-version.sh
./mvnw -s .settings.xml package -DskipTests -T 1C --no-transfer-progress
jfrog rt mvn -s .settings.xml install -Pfull,docs -B --no-transfer-progress
jfrog rt mvn -s .settings.xml install -Pdocs -B --no-transfer-progress
jfrog rt mvn -s .settings.xml install -pl spring-cloud-dataflow-package -B --no-transfer-progress
jfrog rt build-publish
export JFROG_CLI_BUILD_NAME="${JFROG_CLI_BUILD_NAME/spring-cloud-dataflow/spring-cloud-skipper}"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/milestone-worker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ jobs:
jfrog rt build-clean
./spring-cloud-dataflow-package/set-package-version.sh
./mvnw -s .settings.xml package -DskipTests -T 1C --no-transfer-progress
jfrog rt mvn -gs .github/settings.xml -Pstagingmilestone,full,deploymentfiles,docs -B install -DskipTests --no-transfer-progress
jfrog rt mvn -gs .github/settings.xml -Pstagingmilestone,deploymentfiles,docs -B install -DskipTests --no-transfer-progress
jfrog rt build-publish
PROJECT_VERSION=$(./mvnw -gs .github/settings.xml -Pstagingmilestone help:evaluate -Dexpression=project.version -q -DforceStdout)
echo BUILD_ZOO_HANDLER_spring_cloud_dataflow_version=$PROJECT_VERSION >> $GITHUB_ENV
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
echo "::notice ::install"
./mvnw -gs .github/settings.xml -am -pl :spring-cloud-skipper-server-core install -DskipTests -T 1C -B --no-transfer-progress
jfrog rt mvn -gs .github/settings.xml \
-Pstagingrelease,full,deploymentfiles,docs \
-Pstagingrelease,deploymentfiles,docs \
-pl :spring-cloud-dataflow-classic-docs,:spring-cloud-dataflow-docs,:spring-cloud-skipper-docs \
-B install -DskipTests --no-transfer-progress
echo "::notice ::build-publish"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-worker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ jobs:
./spring-cloud-dataflow-package/set-package-version.sh
echo "::notice ::install"
./mvnw -s .settings.xml package -Pstagingrelease -DskipTests -T 1C --no-transfer-progress
jfrog rt mvn -gs .github/settings.xml -Pstagingrelease,full,deploymentfiles,docs -B install -DskipTests --no-transfer-progress
jfrog rt mvn -gs .github/settings.xml -Pstagingrelease,deploymentfiles,docs -B install -DskipTests --no-transfer-progress
echo "::notice ::build-publish"
jfrog rt build-publish
PROJECT_VERSION=$(./mvnw -gs .github/settings.xml -Pstagingrelease help:evaluate -Dexpression=project.version -q -DforceStdout)
Expand Down
1 change: 1 addition & 0 deletions spring-cloud-dataflow-build/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
<sonar.language>java</sonar.language>
<!-- Plugins -->
<maven-compiler-plugin.version>3.13.0</maven-compiler-plugin.version>
<maven-surefire-report-plugin.version>3.5.0</maven-surefire-report-plugin.version>
<maven-eclipse-plugin.version>2.10</maven-eclipse-plugin.version>
<asciidoctorj.version>2.5.7</asciidoctorj.version>
<!-- for backwards compatibility, use maven-checkstyle-plugin.version -->
Expand Down
2 changes: 1 addition & 1 deletion spring-cloud-dataflow-classic-docs/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@
</build>
<profiles>
<profile>
<id>full</id>
<id>docs</id>
<build>
<plugins>
<plugin>
Expand Down
2 changes: 1 addition & 1 deletion spring-cloud-dataflow-docs/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
</dependencies>
<profiles>
<profile>
<id>full</id>
<id>docs</id>
<build>
<plugins>
<plugin>
Expand Down
15 changes: 5 additions & 10 deletions spring-cloud-skipper/README.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,14 @@ To just create the executables, type
$./mvnw clean package -DskipTests -Dmaven.javadoc.skip=true
----

To generate just the RESTDocs documentation

----
./mvnw test -pl spring-cloud-skipper-server-core -Dtest=*Documentation*
----

To build just the documentation, if the RESTDocs generated from tests are already present

To generate just the Javadocs and RESTDocs documentation.
----
./mvnw -DskipTests -Pfull package -pl spring-cloud-skipper-docs
# from root of spring-cloud-dataflow
./mvnw package -Pdocs -DskipTests -f ./spring-cloud-skipper
# from ./spring-cloud-skipper
./mvnw package -Pdocs -DskipTests
----


=== Quick Tour

There are some sample packages in the test directory that you can use to get started. Create a `skipper.yml` file in your home directory
Expand Down
59 changes: 27 additions & 32 deletions spring-cloud-skipper/spring-cloud-skipper-server-core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,14 @@
<argLine>--add-opens java.base/java.util=ALL-UNNAMED</argLine>
<threadCount>1</threadCount>
<forkCount>1</forkCount>
<failIfNoTests>true</failIfNoTests>
<failIfNoTests>false</failIfNoTests>
<includes>
<include>**/*Tests.java</include>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/*Documentation.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
Expand All @@ -279,44 +286,32 @@
</plugins>
</build>
<profiles>
<profile>
<id>full</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>--add-opens java.base/java.util=ALL-UNNAMED</argLine>
<threadCount>1</threadCount>
<forkCount>1</forkCount>
<failIfNoTests>true</failIfNoTests>
<skipTests>false</skipTests>
<includes>
<include>**/*Documentation.java</include>
<include>**/*Tests.java</include>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to include the **/*Tests.java to the docs plugin below. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed to the surefire in docs profile to use the package phase which means test phase executes normal unit tests and package phase generates the snippets.
When -DskipTests is present the normal test phase is skipped. When -Pdocs is not provides then the snippets are not generated.

</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>docs</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<threadCount>1</threadCount>
<forkCount>1</forkCount>
<failIfNoTests>true</failIfNoTests>
<skipTests>false</skipTests>
<includes>
<include>**/*Documentation.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>docs</id>
<goals>
<goal>test</goal>
</goals>
<phase>package</phase>
<configuration>
<argLine>--add-opens java.base/java.util=ALL-UNNAMED</argLine>
<threadCount>1</threadCount>
<forkCount>1</forkCount>
<failIfNoTests>false</failIfNoTests>
<skipTests>false</skipTests>
<includes>
<include>**/*Documentation.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Expand Down
8 changes: 8 additions & 0 deletions ...rc/main/java/org/springframework/cloud/skipper/server/service/PackageMetadataService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -253,9 +253,17 @@ String computeFilename(Resource resource) throws IOException {
if (scheme.equals("file")) {
stringBuilder.append("file");
if (uri.getPath() != null) {
if(uri.getPath().startsWith("/")) {
stringBuilder.append("_root");
}
stringBuilder.append(uri.getPath().replaceAll("/", "_"));
}
else {
if(uri.getSchemeSpecificPart().startsWith("/")) {
stringBuilder.append("_root_");
} else if(!uri.getSchemeSpecificPart().startsWith(".")) {
stringBuilder.append("_");
}
String relativeFilename = uri.getSchemeSpecificPart().replaceAll("^./", "/dot/");
stringBuilder.append(relativeFilename.replaceAll("/", "_"));
}
Expand Down
9 changes: 7 additions & 2 deletions ...-core/src/test/java/org/springframework/cloud/skipper/server/AbstractIntegrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import java.io.File;
import java.io.IOException;
import java.util.List;

import javax.sql.DataSource;

Expand All @@ -33,6 +34,7 @@
import org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.deployer.autoconfigure.ResourceLoadingAutoConfiguration;
import org.springframework.cloud.deployer.spi.app.AppStatus;
import org.springframework.cloud.deployer.spi.local.LocalDeployerAutoConfiguration;
import org.springframework.cloud.skipper.domain.Info;
import org.springframework.cloud.skipper.domain.InstallRequest;
Expand Down Expand Up @@ -139,8 +141,11 @@ protected boolean isDeployed(String releaseName, int releaseVersion) {
Info info = release.getInfo();

logger.info("Status = " + info.getStatus());
return info.getStatus().getStatusCode().equals(StatusCode.DEPLOYED) &&
allAppsDeployed(info.getStatus().getAppStatusList());
boolean deployedStatus = info.getStatus().getStatusCode().equals(StatusCode.DEPLOYED);
List<AppStatus> appStatusList = info.getStatus().getAppStatusList();
logger.info("allAppStatus = " + appStatusList);
boolean allAppsDeployed = allAppsDeployed(appStatusList);
return deployedStatus && allAppsDeployed;
}
catch (Exception e) {
logger.error("Exception getting status", e);
Expand Down
6 changes: 4 additions & 2 deletions ...ver-core/src/test/java/org/springframework/cloud/skipper/server/AbstractMockMvcTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.deployer.autoconfigure.ResourceLoadingAutoConfiguration;
import org.springframework.cloud.deployer.spi.app.AppStatus;
import org.springframework.cloud.skipper.domain.CancelResponse;
import org.springframework.cloud.skipper.domain.Info;
import org.springframework.cloud.skipper.domain.Release;
Expand Down Expand Up @@ -100,8 +101,9 @@ protected boolean isDeployed(String releaseName, int releaseVersion) {
Info info = convertContentToInfo(result.getResponse().getContentAsString());

logger.info("Status = " + info.getStatus());
return info.getStatus().getStatusCode().equals(StatusCode.DEPLOYED) &&
allAppsDeployed(info.getStatus().getAppStatusList());
List<AppStatus> appStatusList = info.getStatus().getAppStatusList();
logger.info("appStatusList = " + appStatusList);
return info.getStatus().getStatusCode().equals(StatusCode.DEPLOYED) && allAppsDeployed(appStatusList);
}
catch (Exception e) {
logger.error("Exception getting status", e);
Expand Down
6 changes: 3 additions & 3 deletions ...test/java/org/springframework/cloud/skipper/server/controller/ReleaseControllerTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ void releaseRollbackAndUndeploy() throws Exception {

// Deploy
String releaseName = "test2";
Release release = install("log", "5.0.0", releaseName);
Release release = install("log", "3.2.1", releaseName);
assertThat(release.getVersion()).isEqualTo(1);

// Check manifest
Expand Down Expand Up @@ -207,7 +207,7 @@ void releaseRollbackAndUndeploy() throws Exception {
@Test
void packageDeployAndUpgrade() throws Exception {
String releaseName = "myLog";
Release release = install("log", "5.0.0", releaseName);
Release release = install("log", "3.2.1", releaseName);
assertThat(release.getVersion()).isEqualTo(1);

// Upgrade
Expand Down Expand Up @@ -250,7 +250,7 @@ void statusReportsErrorForMissingRelease() throws Exception {
void packageUpgradeWithNoDifference() throws Exception {
String releaseName = "myPackage";
String packageName = "log";
String packageVersion = "5.0.0";
String packageVersion = "4.0.0";
Release release = install(packageName, packageVersion, releaseName);
assertThat(release.getVersion()).isEqualTo(1);

Expand Down
2 changes: 1 addition & 1 deletion ...test/java/org/springframework/cloud/skipper/server/controller/docs/BaseDocumentation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ protected Release updateReleaseManifest(Release release) {
protected Release updateReleaseStatus(Release release, StatusCode statusCode) {
Info releaseInfo = new Info();
Status status = new Status();
status.setStatusCode(StatusCode.DELETED);
status.setStatusCode(statusCode);
releaseInfo.setStatus(status);
release.setInfo(releaseInfo);
return release;
Expand Down
3 changes: 2 additions & 1 deletion ...est/java/org/springframework/cloud/skipper/server/controller/docs/DocumentationTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/
package org.springframework.cloud.skipper.server.controller.docs;

import org.junit.jupiter.api.Disabled;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;

Expand All @@ -24,7 +25,7 @@
* @author Ilayaperumal Gopinathan
* @author Corneil du Plessis
*/

@Disabled("we don't need to run suites for docs")
@Suite
@SelectClasses({ AboutDocumentation.class, InstallDocumentation.class, ListDocumentation.class,
CancelDocumentation.class, DeleteDocumentation.class, HistoryDocumentation.class, ManifestDocumentation.class,
Expand Down
3 changes: 2 additions & 1 deletion ...rg/springframework/cloud/skipper/server/controller/docs/RepositoryDocumentationTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/
package org.springframework.cloud.skipper.server.controller.docs;

import org.junit.jupiter.api.Disabled;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;

Expand All @@ -24,7 +25,7 @@
* @author Ilayaperumal Gopinathan
* @author Corneil du Plessis
*/

@Disabled("we don't need to run suites for docs")
@Suite
@SelectClasses({ ApiDocumentation.class, DeployersDocumentation.class, PackageMetadataDocumentation.class,
RepositoriesDocumentation.class, ReleasesDocumentation.class })
Expand Down
Loading
Loading