Skip to content

Commit 86bc73a

Browse files
committed
1.9
- Enhanced docs/examples - Created HWindow interface - Changed default node_modules directory so its inside ./headless-browser/node-js-working-dir
1 parent d128068 commit 86bc73a

18 files changed

+253
-127
lines changed

CONTRIBUTE.md

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#### Beginners
2+
3+
If you have never contributed before, we recommend
4+
this [Beginners Article](https://www.jetbrains.com/help/idea/contribute-to-projects.html). If you are planning to make
5+
big changes, create an issue first, where you explain what you want to do. Thank you in advance for every contribution!
6+
If you don't know how to import a GitHub project, check out this
7+
guide: [IntelliJ IDEA Cloning Guide](https://blog.jetbrains.com/idea/2020/10/clone-a-project-from-github/)
8+
9+
#### Build-Details
10+
11+
- Written in [Java](https://java.com/),
12+
with [JDK 8](https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html), inside
13+
of [IntelliJ IDEA](https://www.jetbrains.com/idea/)
14+
- Built with [Maven](https://maven.apache.org/), profiles: clean package

README.md

+3-38
Original file line numberDiff line numberDiff line change
@@ -33,51 +33,18 @@ Note that the first run may take a bit because Node.js and its modules get insta
3333

3434
(JS = JavaScript; Full Java = If the browser is completely written in Java or not; Downloads = If the browser is able to download files other than html/xml/pdf)
3535

36-
### Contribute/Build
36+
### [Contribute/Build](CONTRIBUTE.md)
3737

38-
#### Beginners
3938

40-
If you have never contributed before, we recommend
41-
this [Beginners Article](https://www.jetbrains.com/help/idea/contribute-to-projects.html). If you are planning to make
42-
big changes, create an issue first, where you explain what you want to do. Thank you in advance for every contribution!
43-
If you don't know how to import a GitHub project, check out this
44-
guide: [IntelliJ IDEA Cloning Guide](https://blog.jetbrains.com/idea/2020/10/clone-a-project-from-github/)
45-
46-
#### Build-Details
47-
48-
- Written in [Java](https://java.com/),
49-
with [JDK 8](https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html), inside
50-
of [IntelliJ IDEA](https://www.jetbrains.com/idea/)
51-
- Built with [Maven](https://maven.apache.org/), profiles: clean package
5239

5340
### Links
5441

5542
- https://spec.whatwg.org/ | Details about JS Web-APIs
5643
- https://www.w3.org/TR/?tag=webapi | Details about JS Web-APIs
5744

5845
### Examples
59-
<details>
60-
<summary>Running Node.js independently</summary>
61-
<pre lang="java">
62-
// Installs Node.js into current working directory if needed
63-
NodeContext ctx = new NodeContext(); // Use another constructor for customization
64-
try{
65-
// Easily install/update needed modules
66-
ctx.npmInstall("name of node module");
67-
68-
// To be able to see the JavaScript code results.
69-
// Otherwise you can also init NodeContext with debugOutput=System.out to achieve this.
70-
ctx.onPrintLine(line -> System.out.println(line);
71-
ctx.executeJavaScript("console.log('hello world!');");
72-
73-
// You can return JavaScript results too.
74-
// Note that you must have a result variable in the provided JS Code for this to work!
75-
String result = ctx.executeJavaScriptAndGetResult("var result = 'my JavaScript result!';");
76-
} catch(Exception e){
77-
e.printStacktrace();
78-
}
79-
</pre>
80-
</details>
46+
- [Running Node.js independently](src/test/java/examples/IndependentNodeJs.java)
47+
- [Customizing browser windows](src/test/java/examples/CustomWindows.java)
8148

8249
### FAQ
8350
<details>
@@ -114,8 +81,6 @@ Thats why this project exists.
11481
We want to provide the latest and best technologies regarding headless browsers and make them available to Java applications.
11582
</details>
11683

117-
![](src/test/java/com/osiris/headlessbrowser/Playground.java)
118-
11984
### Libraries
12085

12186
| Name/Link | Usage | License |

pom.xml

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
<groupId>com.osiris.headlessbrowser</groupId>
88
<artifactId>Headless-Browser</artifactId>
9-
<version>1.8</version>
9+
<version>1.9</version>
1010
<repositories>
1111
<repository>
1212
<id>jitpack</id>

src/main/java/com/osiris/headlessbrowser/HBrowser.java

+10-9
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
package com.osiris.headlessbrowser;
22

33
import com.osiris.headlessbrowser.exceptions.NodeJsCodeException;
4-
import com.osiris.headlessbrowser.windows.GraalWindow;
4+
import com.osiris.headlessbrowser.windows.HWindow;
55
import com.osiris.headlessbrowser.windows.PlaywrightWindow;
66
import com.osiris.headlessbrowser.windows.PuppeteerWindow;
77
import com.osiris.headlessbrowser.windows.WindowBuilder;
@@ -17,12 +17,13 @@ public class HBrowser {
1717

1818
/**
1919
* Creates and returns a new window, built with defaults. <br>
20+
* Remember to close the window either by {@link #closeWindow(HWindow)} or
21+
* by creating the window in a try/catch blocks method. <br>
2022
* By using the {@link WindowBuilder} or the {@link #openCustomWindow()} method <br>
2123
* you can decide between other windows/browsers. <br>
22-
* Since the {@link GraalWindow} has only partial JavaScript support, due to <br>
23-
* currently ongoing Web-APIs implementation, its recommended to use the {@link PuppeteerWindow} instead. <br>
24-
* Its powered by the latest NodeJS-Engine with the help of Puppeteer. <br>
25-
* NodeJS, Puppeteer and Chromiumg get installed into the current working directory automatically (~300mb). <br>
24+
* Some windows are based on Node.js and other Node modules,
25+
* which get installed into the current working directory automatically
26+
* and thus required additional disk space(~300mb). <br>
2627
*/
2728
public PlaywrightWindow openWindow() {
2829
return new WindowBuilder(this).buildPlaywrightWindow();
@@ -44,15 +45,15 @@ public WindowBuilder openCustomWindow() {
4445
}
4546

4647
/**
47-
* Closes the provided {@link GraalWindow}. <br>
48-
* A {@link GraalWindow} can automatically be closed like this:
48+
* Closes the provided {@link HWindow}. <br>
49+
* Note that a {@link HWindow} can automatically be closed like this:
4950
* <pre>
5051
* try(HWindow hWindow = openNewWindow()){
5152
* // Do stuff here...
52-
* } // Window gets automatically closed when leaving the try/catch block.
53+
* } // Gets automatically closed when leaving the try/catch block.
5354
* </pre>
5455
*/
55-
public void closeWindow(PuppeteerWindow window) throws Exception {
56+
public void closeWindow(HWindow window) throws Exception {
5657
window.close();
5758
}
5859
}

src/main/java/com/osiris/headlessbrowser/js/contexts/NodeContext.java

+51-31
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,9 @@ public class NodeContext implements AutoCloseable {
3030
private final PrintStream debugOutput;
3131
private final File lastJsCodeExecutionResultFile;
3232
private final int timeout;
33+
private final File parentNodeDir;
3334
private final File installationDir;
35+
private final File workingDir;
3436
private final File nodeExeFile;
3537
private final File npmExeFile;
3638
private final File npxExeFile;
@@ -42,28 +44,28 @@ public NodeContext() {
4244
}
4345

4446
/**
45-
* @param installationDir path of an empty directory (can also not exist yet) to install the latest Node.js into if needed.
46-
* If null Node.js will get installed into ./NodeJS-Installation ("." is the current working directory).
47-
* @param debugOutput if null, debug output won't be written/printed, otherwise it gets printed/written to the provided {@link OutputStream}.
48-
* @param timeout the max time in seconds to wait for JavaScript code to finish. Set to 0 to disable.
47+
* @param parentNodeDir path of an empty directory (can also not exist yet) to install the latest Node.js into if needed.
48+
* If null Node.js will get installed into ./NodeJS-Installation ("." is the current working directory).
49+
* @param debugOutput if null, debug output won't be written/printed, otherwise it gets printed/written to the provided {@link OutputStream}.
50+
* @param timeout the max time in seconds to wait for JavaScript code to finish. Set to 0 to disable.
4951
*/
50-
public NodeContext(File installationDir, OutputStream debugOutput, int timeout) {
52+
public NodeContext(File parentNodeDir, OutputStream debugOutput, int timeout) {
5153
this.timeout = timeout;
5254
if (debugOutput == null)
5355
this.debugOutput = new PrintStream(new TrashOutput());
5456
else
5557
this.debugOutput = new PrintStream(debugOutput);
5658
PrintStream out = this.debugOutput;
57-
if (installationDir == null) {
58-
this.installationDir = new File(System.getProperty("user.dir") + "/NodeJS-Installation");
59-
installationDir = this.installationDir;
59+
if (parentNodeDir == null) {
60+
this.parentNodeDir = new File(System.getProperty("user.dir") + "/NodeJS-Installation");
61+
parentNodeDir = this.parentNodeDir;
6062
} else
61-
this.installationDir = installationDir;
63+
this.parentNodeDir = parentNodeDir;
6264
// Download and install NodeJS into current working directory if no installation found
6365
try {
6466
determineArchAndOs();
65-
if (!installationDir.exists()
66-
|| Objects.requireNonNull(installationDir.listFiles()).length == 0) {
67+
if (!parentNodeDir.exists()
68+
|| Objects.requireNonNull(parentNodeDir.listFiles()).length == 0) {
6769
String url = "https://nodejs.org/dist/latest/";
6870
out.println("Installing latest NodeJS release from '" + url + "'...");
6971
Document docLatest = Jsoup.connect(url).get();
@@ -82,9 +84,9 @@ public NodeContext(File installationDir, OutputStream debugOutput, int timeout)
8284
throw new FileNotFoundException("Failed to find latest NodeJS download url at '" + url + "' for OS '" + osType.name() + "' with ARCH '" + osArchitectureType.name() + "'.");
8385

8486
// Download the zip file and extract its contents
85-
if (!installationDir.exists()) installationDir.mkdirs();
87+
if (!parentNodeDir.exists()) parentNodeDir.mkdirs();
8688
if (osType.equals(OperatingSystemType.WINDOWS)) {
87-
File downloadZip = new File(installationDir + "/download.zip");
89+
File downloadZip = new File(parentNodeDir + "/download.zip");
8890
if (downloadZip.exists()) downloadZip.delete();
8991
downloadZip.createNewFile();
9092
DownloadTask downloadTask = new DownloadTask("Download", new BetterThreadManager(), downloadUrl, downloadZip, "zip");
@@ -95,14 +97,14 @@ public NodeContext(File installationDir, OutputStream debugOutput, int timeout)
9597
}
9698
out.println("Download-Task > " + downloadTask.getStatus());
9799

98-
out.print("Extracting NodeJS files...");
100+
out.print("Extracting Node.js files...");
99101
out.flush();
100102
ZipFile zipFile = new ZipFile(downloadZip);
101-
zipFile.extractFile(zipFile.getFileHeaders().get(0).getFileName(), installationDir.getPath());
103+
zipFile.extractFile(zipFile.getFileHeaders().get(0).getFileName(), parentNodeDir.getPath());
102104
downloadZip.delete();
103105
out.println(" SUCCESS!");
104106
} else {
105-
File downloadFile = new File(installationDir + "/download.tar.gz");
107+
File downloadFile = new File(parentNodeDir + "/download.tar.gz");
106108
if (downloadFile.exists()) downloadFile.delete();
107109
downloadFile.createNewFile();
108110
DownloadTask downloadTask = new DownloadTask("Download", new BetterThreadManager(), downloadUrl, downloadFile, "gzip", "tar.gz", "tar", "tar+gzip", "x-gtar", "x-gzip", "x-tgz");
@@ -112,31 +114,37 @@ public NodeContext(File installationDir, OutputStream debugOutput, int timeout)
112114
Thread.sleep(1000);
113115
}
114116
out.println("Download-Task > " + downloadTask.getStatus());
115-
out.print("Extracting NodeJS files...");
117+
out.print("Extracting Node.js files...");
116118
out.flush();
117-
ArchiverFactory.createArchiver(downloadFile).extract(downloadFile, installationDir);
119+
ArchiverFactory.createArchiver(downloadFile).extract(downloadFile, parentNodeDir);
118120
// Result should be .../headless-browser/node-js/node<version>/...
119121
downloadFile.delete();
120122
out.println(" SUCCESS!");
121123
}
122124
}
123125

126+
this.installationDir = this.parentNodeDir.listFiles()[0];
127+
Objects.requireNonNull(installationDir);
128+
this.workingDir = new File(this.parentNodeDir.getParentFile() + "/node-js-working-dir");
129+
Objects.requireNonNull(workingDir);
130+
if (!workingDir.exists()) workingDir.mkdirs();
131+
124132
if (osType.equals(OperatingSystemType.WINDOWS)) {
125-
nodeExeFile = new File(installationDir.listFiles()[0] + "/node.exe");
126-
npmExeFile = new File(installationDir.listFiles()[0] + "/npm.cmd");
127-
npxExeFile = new File(installationDir.listFiles()[0] + "/npx.cmd");
133+
nodeExeFile = new File(installationDir + "/node.exe");
134+
npmExeFile = new File(installationDir + "/npm.cmd");
135+
npxExeFile = new File(installationDir + "/npx.cmd");
128136
} else { // Linux, mac and co.
129-
nodeExeFile = new File(installationDir.listFiles()[0] + "/bin/node");
130-
npmExeFile = new File(installationDir.listFiles()[0] + "/bin/npm");
131-
npxExeFile = new File(installationDir.listFiles()[0] + "/bin/npx");
137+
nodeExeFile = new File(installationDir + "/bin/node");
138+
npmExeFile = new File(installationDir + "/bin/npm");
139+
npxExeFile = new File(installationDir + "/bin/npx");
132140
}
133141

134142
if (!nodeExeFile.exists())
135-
throw new Exception("node.exe couldn't be found in " + installationDir.listFiles()[0]);
143+
throw new Exception("node.exe couldn't be found in " + installationDir);
136144
if (!npmExeFile.exists())
137-
throw new Exception("npm.cmd couldn't be found in " + installationDir.listFiles()[0]);
145+
throw new Exception("npm.cmd couldn't be found in " + installationDir);
138146
if (!npxExeFile.exists())
139-
throw new Exception("npx.cmd couldn't be found in " + installationDir.listFiles()[0]);
147+
throw new Exception("npx.cmd couldn't be found in " + installationDir);
140148
} catch (Exception e) {
141149
System.err.println("Error during installation of NodeJS. Details:");
142150
throw new RuntimeException(e);
@@ -148,6 +156,7 @@ public NodeContext(File installationDir, OutputStream debugOutput, int timeout)
148156
out.flush();
149157
ProcessBuilder processBuilder = new ProcessBuilder(Arrays.asList(
150158
nodeExeFile.getAbsolutePath(), "--interactive"));
159+
processBuilder.directory(workingDir);
151160
// Must be inherited so that NodeJS closes when this application closes.
152161
// Wrong! It seems like NodeJS closes if the parent process dies, even if its Piped I/O.
153162
process = processBuilder.start();
@@ -536,7 +545,7 @@ public Process executeNpmWithArgs(String... args) throws IOException, Interrupte
536545
List<String> commands = new ArrayList<>();
537546
commands.add(npmExeFile.getAbsolutePath());
538547
commands.addAll(Arrays.asList(args));
539-
Process process = new ProcessBuilder(commands).start();
548+
Process process = new ProcessBuilder(commands).directory(workingDir).start();
540549
new AsyncInputStream(process.getInputStream()).listeners.add(line -> debugOutput.println("[NPM] " + line));
541550
new AsyncInputStream(process.getErrorStream()).listeners.add(line -> System.err.println("[NPM-ERROR] " + line));
542551
while (process.isAlive())
@@ -549,7 +558,7 @@ public Process executeNpxWithArgs(String... args) throws IOException, Interrupte
549558
List<String> commands = new ArrayList<>();
550559
commands.add(npxExeFile.getAbsolutePath());
551560
commands.addAll(Arrays.asList(args));
552-
Process process = new ProcessBuilder(commands).start();
561+
Process process = new ProcessBuilder(commands).directory(workingDir).start();
553562
new AsyncInputStream(process.getInputStream()).listeners.add(line -> debugOutput.println("[NPX] " + line));
554563
new AsyncInputStream(process.getErrorStream()).listeners.add(line -> System.err.println("[NPX-ERROR] " + line));
555564
while (process.isAlive())
@@ -566,8 +575,8 @@ public NodeContext onPrintLine(Consumer<String> listener) {
566575
return this;
567576
}
568577

569-
public File getInstallationDir() {
570-
return installationDir;
578+
public File getParentNodeDir() {
579+
return parentNodeDir;
571580
}
572581

573582
public File getNodeExeFile() {
@@ -610,6 +619,17 @@ public OperatingSystemType getOsType() {
610619
return osType;
611620
}
612621

622+
public File getWorkingDir() {
623+
return workingDir;
624+
}
625+
626+
public File getNpmExeFile() {
627+
return npmExeFile;
628+
}
629+
630+
public File getNpxExeFile() {
631+
return npxExeFile;
632+
}
613633

614634
public enum OperatingSystemArchitectureType {
615635
X64("x64"),

src/main/java/com/osiris/headlessbrowser/utils/UtilsChrome.java

+1
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
public class UtilsChrome {
99

1010
/**
11+
* <h1>WORK IN PROGRESS!</h1> <br>
1112
* Generates the 'add_missing_objects.js' script in your current working dir. <br>
1213
* The script is meant to be run in the headless window/page instance. <br>
1314
* It will add all Json objects that are missing to that window/page <br>

src/main/java/com/osiris/headlessbrowser/windows/GraalWindow.java

+6-2
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,11 @@
1414
import java.util.Map;
1515

1616
/**
17-
* Headless-Window.
17+
* Headless-Window with GraalJS as JavaScript engine.
1818
*
1919
* @author Osiris-Team
2020
*/
21-
public class GraalWindow implements AutoCloseable {
21+
public class GraalWindow implements HWindow {
2222
private final GraalContext graalContext = new GraalContext(this);
2323
private HBrowser parentBrowser;
2424
private boolean enableJavaScript;
@@ -27,6 +27,10 @@ public class GraalWindow implements AutoCloseable {
2727
private String authority;
2828
private String javaScriptCode;
2929

30+
/**
31+
* <p style="color: red;">Note that this is not the recommended way of creating a NodeWindow object.</p>
32+
* Use the {@link WindowBuilder} instead. The {@link HBrowser} has a shortcut method for creating custom windows: {@link HBrowser#openCustomWindow()}.
33+
*/
3034
public GraalWindow(HBrowser parentBrowser, boolean enableJavaScript, Map<String, String> customHeaders) {
3135
this.parentBrowser = parentBrowser;
3236
this.enableJavaScript = enableJavaScript;
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
package com.osiris.headlessbrowser.windows;
2+
3+
public interface HWindow extends AutoCloseable {
4+
5+
}

src/main/java/com/osiris/headlessbrowser/windows/LightWindow.java

+6-2
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@
2020
import java.util.Map;
2121

2222
/**
23-
* Headless-Window with Node.js as JavaScript engine and Node modules replacing the regular web-apis.
23+
* Headless-Window with Node.js/V8 as JavaScript engine and Node modules replacing the regular web-apis.
2424
*
2525
* @author Osiris-Team
2626
*/
27-
public class LightWindow implements AutoCloseable {
27+
public class LightWindow implements HWindow {
2828
private final NodeContext jsContext;
2929
private final OutputStream debugOutput;
3030
private final HBrowser parentBrowser;
@@ -39,6 +39,10 @@ public class LightWindow implements AutoCloseable {
3939
private String authority;
4040
private String javaScriptCode;
4141

42+
/**
43+
* <p style="color: red;">Note that this is not the recommended way of creating a NodeWindow object.</p>
44+
* Use the {@link WindowBuilder} instead. The {@link HBrowser} has a shortcut method for creating custom windows: {@link HBrowser#openCustomWindow()}.
45+
*/
4246
public LightWindow(HBrowser parentBrowser, boolean enableJavaScript, Map<String, String> customHeaders,
4347
OutputStream debugOutput, boolean isHeadless, File userDataDir, boolean isDevTools, int jsTimeout, boolean makeUndetectable) {
4448
this.parentBrowser = parentBrowser;

0 commit comments

Comments
 (0)