Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Knowage 8043 #891

Merged
merged 22 commits into from
Sep 20, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
da96b6a
Revert "Possible Bugfix KNOWAGE-8043"
BojanSovticEngIT Aug 29, 2023
997e9ab
Merge remote-tracking branch 'upstream/master' into KNOWAGE-8043
BojanSovticEngIT Sep 6, 2023
e3c6a3c
Bugfix KNOWAGE-8043
BojanSovticEngIT Sep 6, 2023
aaa3d09
Merge remote-tracking branch 'upstream/master' into KNOWAGE-8043
BojanSovticEngIT Sep 13, 2023
8a66b78
Bugfix KNOWAGE-8043
BojanSovticEngIT Sep 13, 2023
09ea635
Bugfixes KNOWAGE-8043
BojanSovticEngIT Sep 18, 2023
fd75ee0
Update DossierActivityResource.java
BojanSovticEngIT Sep 18, 2023
c560439
Update SelfServiceDataSetCRUD.java
BojanSovticEngIT Sep 18, 2023
0fab752
Bugfix KNOWAGE-8043
BojanSovticEngIT Sep 19, 2023
2284ba0
Update WorksRepository.java
BojanSovticEngIT Sep 19, 2023
f302cdb
Update DossierActivityResource.java
BojanSovticEngIT Sep 19, 2023
b168ec2
Update Utils.java
BojanSovticEngIT Sep 19, 2023
9bb7574
Update SDKObjectsConverter.java
BojanSovticEngIT Sep 19, 2023
71231ad
Update SelfServiceDataSetCRUD.java
BojanSovticEngIT Sep 19, 2023
b7ecf4e
Update DossierActivityResource.java
BojanSovticEngIT Sep 19, 2023
c0dc7cc
Fixes
BojanSovticEngIT Sep 19, 2023
f0f1db5
PathTraversal fixes
BojanSovticEngIT Sep 19, 2023
45a0eff
Update PathTraversalChecker.java
BojanSovticEngIT Sep 19, 2023
6520e49
Merge remote-tracking branch 'upstream/master' into KNOWAGE-8043
BojanSovticEngIT Sep 19, 2023
ca94b0d
Bugfix Knowage-8043
BojanSovticEngIT Sep 19, 2023
fe68d3d
Bugfix Knowage-8043
BojanSovticEngIT Sep 20, 2023
6d9fa38
Update ManagePreviewFileAction.java
BojanSovticEngIT Sep 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import org.json.JSONException;
import org.json.JSONObject;

import it.eng.knowage.commons.security.PathTraversalChecker;
import it.eng.spagobi.commons.SingletonConfig;
import it.eng.spagobi.commons.bo.UserProfile;
import it.eng.spagobi.commons.services.AbstractSpagoBIAction;
Expand Down Expand Up @@ -107,7 +108,7 @@ public void doService() {
// checks for path traversal attacks
private void checkRequiredFile(String fileName) {
File targetDirectory = GeneralUtilities.getPreviewFilesStorageDirectoryPath();
FileUtils.checkPathTraversalAttack(fileName, targetDirectory);
PathTraversalChecker.get(fileName, targetDirectory.getName());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't make sense: inputs are reversed, since targetDirectory is the safe dir and filename is what has to be checked

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

}

private JSONObject uploadFile() throws Exception {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
import java.io.File;
import java.io.FileOutputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
Expand Down Expand Up @@ -131,16 +130,15 @@ public Response getresourcePath(@QueryParam("templateName") String fileName, @Qu
String separator = File.separator;
if (fileName.endsWith("?"))
fileName = fileName.substring(0, fileName.length() - 1);

java.nio.file.Path outputPath = Paths.get(SpagoBIUtilities.getResourcePath(), "dossier", String.valueOf(documentId), fileName);
File file = outputPath.toFile();
String safeDirectory = SpagoBIUtilities.getResourcePath() + separator + "dossier" + separator + documentId;
PathTraversalChecker.get(safeDirectory, fileName);
String outPath = safeDirectory + separator + fileName;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge them in 1 line: PathTraversalChecker.get has to return a File or something similar

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

ResponseBuilder responseBuilder = null;

File file = new File(outPath);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge this with the above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

JSONObject response = new JSONObject();
File dossierDir = new File(SpagoBIUtilities.getResourcePath() + separator + "dossier" + separator + documentId + separator);
try {
PathTraversalChecker.isValidFileName(fileName);
PathTraversalChecker.preventPathTraversalAttack(file, dossierDir);
PathTraversalChecker.get(fileName, dossierDir.getName());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inputs reversed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

byte[] bytes = Files.readAllBytes(file.toPath());
responseBuilder = Response.ok(bytes);
responseBuilder.header("Content-Disposition", "attachment; filename=" + fileName);
Expand Down Expand Up @@ -171,8 +169,7 @@ public Response checkPathFile(@QueryParam("templateName") String fileName, @Quer
File dossierDir = new File(SpagoBIUtilities.getResourcePath() + separator + "dossier" + separator + documentId + separator);
JSONObject response = new JSONObject();
try {
PathTraversalChecker.isValidFileName(fileName);
PathTraversalChecker.preventPathTraversalAttack(file, dossierDir);
PathTraversalChecker.get(fileName, dossierDir.getName());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inputs reversed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Files.readAllBytes(file.toPath());
response.put("STATUS", "OK");
} catch (Exception e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1621,9 +1621,12 @@ private void renameAndMoveDatasetFile(String originalFileName, String newFileNam
String filePath = resourcePath + File.separatorChar + "dataset" + File.separatorChar + "files" + File.separatorChar + "temp" + File.separatorChar;
String fileNewPath = resourcePath + File.separatorChar + "dataset" + File.separatorChar + "files" + File.separatorChar;

PathTraversalChecker.get(filePath, originalFileName);
File originalDatasetFile = new File(filePath + originalFileName);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge those lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

PathTraversalChecker.get(fileNewPath, newFileName + "." + fileType.toLowerCase());
File newDatasetFile = new File(fileNewPath + newFileName + "." + fileType.toLowerCase());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge those lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

if (originalDatasetFile.exists()) {

if (newDatasetFile != null && originalDatasetFile != null && originalDatasetFile.exists()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how files can be null

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

/*
* This method copies the contents of the specified source file to the specified destination file. The directory holding the destination file is
* created if it does not exist. If the destination file exists, then this method will overwrite it.
Expand Down Expand Up @@ -1741,8 +1744,10 @@ private JSONObject getCkanDataSetConfig(SelfServiceDataSetDTO selfServiceDataSet
private void deleteDatasetFile(String fileName, String resourcePath, String fileType) {
String filePath = resourcePath + File.separatorChar + "dataset" + File.separatorChar + "files" + File.separatorChar + "temp" + File.separatorChar;

PathTraversalChecker.get(filePath, fileName);
File datasetFile = new File(filePath + fileName);
if (datasetFile.exists()) {

if (datasetFile != null && datasetFile.exists()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how files can be null

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

datasetFile.delete();
}
}
Expand Down Expand Up @@ -1796,10 +1801,7 @@ private Integer getCategoryCode(String category) {

try {
ICategoryDAO categoryDao = DAOFactory.getCategoryDAO();
categories = categoryDao.getCategoriesForDataset()
.stream()
.map(Domain::fromCategory)
.collect(toList());
categories = categoryDao.getCategoriesForDataset().stream().map(Domain::fromCategory).collect(toList());
} catch (Throwable t) {
throw new SpagoBIRuntimeException("An unexpected error occured while loading categories types from database", t);
}
Expand Down Expand Up @@ -2174,10 +2176,7 @@ protected List<Integer> getCategories(IEngUserProfile profile) {
ICategoryDAO categoryDao = DAOFactory.getCategoryDAO();

// TODO : Makes sense?
List<Domain> dialects = categoryDao.getCategoriesForDataset()
.stream()
.map(Domain::fromCategory)
.collect(toList());
List<Domain> dialects = categoryDao.getCategoriesForDataset().stream().map(Domain::fromCategory).collect(toList());
if (dialects == null || dialects.size() == 0) {
return null;
}
Expand All @@ -2191,10 +2190,7 @@ protected List<Integer> getCategories(IEngUserProfile profile) {

List<RoleMetaModelCategory> aRoleCategories = roledao.getMetaModelCategoriesForRole(role.getId());
List<RoleMetaModelCategory> resp = new ArrayList<>();
List<Domain> array = categoryDao.getCategoriesForDataset()
.stream()
.map(Domain::fromCategory)
.collect(toList());
List<Domain> array = categoryDao.getCategoriesForDataset().stream().map(Domain::fromCategory).collect(toList());
for (RoleMetaModelCategory r : aRoleCategories) {
for (Domain dom : array) {
if (r.getCategoryId().equals(dom.getValueId())) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.apache.log4j.Logger;
import org.json.JSONObject;

import it.eng.knowage.commons.security.PathTraversalChecker;
import it.eng.qbe.dataset.QbeDataSet;
import it.eng.spago.base.SourceBean;
import it.eng.spago.base.SourceBeanException;
Expand Down Expand Up @@ -228,8 +229,11 @@ public ObjTemplate fromSDKTemplateToObjTemplate(SDKTemplate sdkTemplate) {
}
if (dh != null && dh.getName() != null) {
LOGGER.debug("Deleting attachment file ...");
File attachment = new File(dh.getName());
if (attachment.exists() && attachment.isFile()) {
String fileName = dh.getName();
PathTraversalChecker.isValidFileName(fileName);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fileName seems to be a complete path instead of a single file name. Please check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to talk about this in today's call.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert the changes, most likely this is a false positive

File attachment = new File(fileName);

if (attachment != null && attachment.exists() && attachment.isFile()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how files can be null

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

boolean attachmentFileDeleted;
try {
attachmentFileDeleted = Files.deleteIfExists(attachment.toPath());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@

import org.apache.log4j.Logger;

import it.eng.knowage.commons.security.PathTraversalChecker;
import it.eng.spago.security.IEngUserProfile;
import it.eng.spagobi.services.proxy.DocumentExecuteServiceProxy;
import it.eng.spagobi.utilities.mime.MimeUtils;
Expand Down Expand Up @@ -78,16 +79,9 @@ public void service(HttpServletRequest request, HttpServletResponse response) {
imageFile = new File(completeImageFileName);

File parent = imageFile.getParentFile();
// Prevent directory traversal (path traversal) attacks
if (!imageTmpDir.equals(parent)) {
logger.error("Trying to access the file [" + imageFile.getAbsolutePath() + "] that is not inside ${java.io.tmpdir}/birt!!!");
throw new SecurityException("Trying to access the file [" + imageFile.getAbsolutePath() + "] that is not inside ${java.io.tmpdir}/birt!!!");
}
String fileName = imageFile.toString();

if (!imageFile.exists()) {
logger.error("File " + imageFile.getPath() + " not found");
return;
}
davide-zerbetto marked this conversation as resolved.
Show resolved Hide resolved
PathTraversalChecker.get(parent.toString(), fileName);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parent is NOT a safe dir, imageDirectory or imageTmpDir are safe.
Filename is a complete path, don't see how PathTraversalChecker.get will work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to talk about this in today's call.


try {
fis = new FileInputStream(imageFile);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@

import org.apache.log4j.Logger;

import it.eng.knowage.commons.security.PathTraversalChecker;
import it.eng.spagobi.engines.birt.BirtReportServlet;

public class Utils {
Expand All @@ -36,8 +37,7 @@ public class Utils {
/**
* Resolve system properties.
*
* @param logDir
* the log dir
* @param logDir the log dir
*
* @return the string
*/
Expand All @@ -54,10 +54,8 @@ public static String resolveSystemProperties(String logDir) {
/**
* Resolve system properties.
*
* @param logDir
* the log dir
* @param startIndex
* the start index
* @param logDir the log dir
* @param startIndex the start index
*
* @return the string
*/
Expand Down Expand Up @@ -89,14 +87,10 @@ public static void sendPage(HttpServletResponse response, int pageNumber, String
String completeImageFileName = null;
String mimeType = "text/html";

htmlFile = new File(BirtReportServlet.OUTPUT_FOLDER + File.separator + reportExecutionId, BirtReportServlet.PAGE_FILE_NAME + pageNumber + ".html");

// file path traversal security check
if (!((htmlFile.getParentFile().getParent() + File.separator).equals(BirtReportServlet.OUTPUT_FOLDER))) {
logger.error("Security exception: parent folder " + htmlFile.getParent() + " is not equal to expected folder " + BirtReportServlet.OUTPUT_FOLDER);
throw new RuntimeException(
"Security exception: parent folder " + htmlFile.getParent() + " is not equal to expected folder " + BirtReportServlet.OUTPUT_FOLDER);
}
String directory = BirtReportServlet.OUTPUT_FOLDER + File.separator + reportExecutionId;
String fileName = BirtReportServlet.PAGE_FILE_NAME + pageNumber + ".html";
PathTraversalChecker.get(fileName, directory);
htmlFile = new File(directory, fileName);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge those 2 lines

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use
File htmlFile = PathTraversalChecker.get(BirtReportServlet.OUTPUT_FOLDER, reportExecutionId, fileName);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


try (InputStream fis = new FileInputStream(htmlFile);) {
writeToOutput(response, mimeType, fis);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@

import com.google.gson.Gson;

import it.eng.knowage.commons.security.PathTraversalChecker;
import it.eng.knowage.engine.cockpit.api.export.AbstractFormatExporter;
import it.eng.knowage.engine.cockpit.api.export.ExporterClient;
import it.eng.knowage.engine.cockpit.api.export.excel.exporters.IWidgetExporter;
Expand Down Expand Up @@ -149,7 +150,11 @@ public byte[] getBinaryData(String documentLabel) throws IOException, Interrupte
}

private byte[] getByteArrayFromFile(Path excelFile, Path outputDir) {
try (FileInputStream fis = new FileInputStream(excelFile.toString()); ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
String fileName = excelFile.toString();

PathTraversalChecker.isValidFileName(fileName);

try (FileInputStream fis = new FileInputStream(fileName); ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
byte[] buf = new byte[1024];
for (int readNum; (readNum = fis.read(buf)) != -1;) {
// Writes len bytes from the specified byte array starting at offset off to this byte array output stream
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@

import org.apache.log4j.Logger;

import it.eng.knowage.commons.security.PathTraversalChecker;

public class WorksRepository {
private static final Logger LOGGER = Logger.getLogger(WorksRepository.class);

Expand Down Expand Up @@ -68,7 +70,6 @@ public File getExecutableWorkProjectDir(CommonjWork work) {
return worksDir;
}


/**
* Gets the executable work dir.
*
Expand All @@ -78,12 +79,15 @@ public File getExecutableWorkProjectDir(CommonjWork work) {
*/
public File getExecutableWorkDir(CommonjWork work) {
LOGGER.debug("IN");
File workDir = new File(rootDir, work.getWorkName());

String fileName = work.getWorkName();
PathTraversalChecker.get(rootDir.getName(), fileName);
File workDir = new File(rootDir, fileName);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge those 2 lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


LOGGER.debug("OUT");
return workDir;
}


/**
* Gets the executable work file.
*
Expand All @@ -103,9 +107,12 @@ public File getExecutableWorkFile(CommonjWork work) {
* @return true, if successful
*/
public boolean containsWork(CommonjWork work) {
String fileName = work.getWorkName();

PathTraversalChecker.get(rootDir.getName(), fileName);
File workFolder = new File(rootDir, fileName);

File workFolder=new File(rootDir, work.getWorkName());
return workFolder.exists();
return workFolder != null && workFolder.exists();
}

}
Loading