Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Q20-15 - story/Q20-15 #33

Open
wants to merge 69 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
b3a7d40
[Q20-15] - Model for find pattern api request
MaulikLim Jun 15, 2022
eac4100
[Q20-15] - Classes for holding the pattern graph
MaulikLim Jun 16, 2022
77a175b
[Q20-15] - Building pattern graph
MaulikLim Jun 17, 2022
7aeff7a
[Q20-15] - Finding patterns in the graph
MaulikLim Jun 20, 2022
2297cc0
[Q20-15] - Created models for holding response from conceptpower
MaulikLim Jun 21, 2022
5d940c8
[Q20-15] - Parent conceptpower response class
MaulikLim Jun 22, 2022
c1d622b
[Q20-15] - Conceptpower Connector Interface
MaulikLim Jun 22, 2022
ce2e3fa
[Q20-15] - Conceptpower connector
MaulikLim Jun 23, 2022
274d9b4
[Q20-15] - extracting subgraphs that match the pattern
MaulikLim Jun 24, 2022
9ccc289
[Q20-15] - mapping sub-graphs to triples
MaulikLim Jun 27, 2022
26e079d
[Q20-15] - match pattern interpretation with alternative ids of a gra…
MaulikLim Jun 28, 2022
3c0fff9
[Q20-15] - create mapped triple group if not found
MaulikLim Jun 29, 2022
47ea50c
[Q20-15] - get by id when interpretation contains term 'digitalhps' a…
MaulikLim Jun 30, 2022
8eaaf88
[Q20-15] - iterate over all event graphs for a collection and search …
MaulikLim Jul 5, 2022
ee62a04
[Q20-15] - code refactoring
MaulikLim Jul 6, 2022
bab0acb
[Q20-15] - asynchronous processing for patterns
MaulikLim Jul 7, 2022
cfcd8af
[Q20-15] - restructured the code for job creation and management
MaulikLim Jul 8, 2022
03ebd67
[Q20-15] - correcting minor mistake
MaulikLim Jul 11, 2022
3b28ae4
[Q20-15] - bug fixes - error handling
MaulikLim Jul 12, 2022
c0852d9
[Q20-15] - minor corrections
MaulikLim Jul 13, 2022
452c08e
[Q20-15] - code cleanup, javadocs, renamed classes
MaulikLim Jul 13, 2022
7211aa9
[Q20-15] - changing response details for map triples API
MaulikLim Jul 14, 2022
5abf144
[Q20-15] - configuring quadriga uris for response
MaulikLim Jul 18, 2022
6200570
[Q20-15] - respository query correction
MaulikLim Jul 20, 2022
f461cec
[Q20-15] Added mapGraphToPattern controller
ShivaniYerram Apr 18, 2023
c4e0d03
[Q20-15]Tested response from job api.
ShivaniYerram Apr 24, 2023
481cee5
[Q20-15] Added test cases for MapGraphToTripleController and AddNetwo…
ShivaniYerram Apr 25, 2023
362cb65
[Q20-15] Added test cases for MapGraphToTripleController and AddNetwo…
ShivaniYerram Apr 26, 2023
5a22df9
[Q20-15] Added testcases for MapGraphToTriple and AddNetworkApi contr…
ShivaniYerram May 8, 2023
392d85c
[Q20-15] Added Test cases for controllers.
ShivaniYerram May 25, 2023
e91de9e
[Q20-15]
ShivaniYerram Jun 5, 2023
5666077
[Q20-15]
ShivaniYerram Jun 5, 2023
344f59b
[Q20-15] Removed client id and secret from pom.
ShivaniYerram Jun 5, 2023
5defb60
[Q20-15] Resolved code factor issue.
ShivaniYerram Jun 7, 2023
bcb1b52
[Q20-15] Resolved merge conflicts.
ShivaniYerram Jun 7, 2023
2223f0c
[Q20-15] resolved merge conflicts.
ShivaniYerram Jun 8, 2023
0b3606f
[Q20-15] Addressed review comments.
ShivaniYerram Oct 26, 2023
d809c7f
[Q20-15] Addressed Review comments.
ShivaniYerram Oct 27, 2023
eee20d0
[Q20-15] Addressed review comments.
ShivaniYerram Nov 2, 2023
2b545e2
[Q20-15] Added custom mapping to mapped triple type enum, checked for…
ShivaniYerram Nov 20, 2023
1165908
[Q20-15] Looked into failing job, added test class for asyncpattern p…
ShivaniYerram Nov 21, 2023
93b0d29
[Q20-15] Looked into failing job, added test class for asyncpattern p…
ShivaniYerram Nov 21, 2023
b132377
[Q20-15] Checked by changing the request and mapping.
ShivaniYerram Nov 22, 2023
a56968a
[Q20-15] Resolved errors in AsyncPatternProcessorImplTest.
ShivaniYerram Nov 28, 2023
967569b
[Q20-15] Added test methods for asyncpatternprocessorimpl.
ShivaniYerram Nov 30, 2023
cd77e32
[Q20-15] Working on review comments.
ShivaniYerram Feb 8, 2024
3e5b4f6
[Q20-15] Addressed review comments.
ShivaniYerram Feb 9, 2024
89a902d
[Q20-15]Added javadoc and changed threads to asyncPatternProcessor.
ShivaniYerram Feb 12, 2024
08411d9
[Q20-15] Worked on async job processing, facing pending status on job…
ShivaniYerram Feb 14, 2024
814bdc9
[Q20-15] Worked on review comments regarding Custom mappings, separat…
ShivaniYerram Feb 19, 2024
5af8a7e
[Q20-15] Addressed review comments.
ShivaniYerram Feb 20, 2024
40f882b
[Q20-15] Resolved merge conflicts.
ShivaniYerram Feb 20, 2024
5c19c21
[Q20-15] Resolved code factor issue.
ShivaniYerram Feb 20, 2024
4d5f37f
[Q20-15] Addressed review comments.
ShivaniYerram Feb 28, 2024
e462496
[Q20-15] Addressed review comments, should wait for Q20-16 to be merg…
ShivaniYerram Feb 28, 2024
53cb294
[Q20-15] Addressed review comments.
ShivaniYerram Apr 8, 2024
e185877
[Q20-15] Addressed review comments.
ShivaniYerram Apr 10, 2024
e249731
[Q20-15] Resolved code factor: Placing overloaded methods together.
ShivaniYerram Apr 10, 2024
6c7a5fe
[Q20-15] removed pom.xml change
pooja-thalur Sep 6, 2024
86a25d3
[Q20-15] config changes
pooja-thalur Sep 10, 2024
19cdda1
[Q20-15] moving business logic from controller
pooja-thalur Sep 10, 2024
814fb46
[Q20-15] addressed review comments
pooja-thalur Sep 13, 2024
83e2a0e
[Q20-15] review comments addressed and testcases issue fixed
pooja-thalur Sep 17, 2024
f71b945
[Q20-15] codefactor issue
pooja-thalur Sep 18, 2024
c26eb0f
[Q20-15] addressed review comments
pooja-thalur Sep 18, 2024
519fcf6
[Q20-15] removed spaces and empty lines
pooja-thalur Sep 18, 2024
06d4998
[Q20-15] fixing codefactor issue
pooja-thalur Sep 18, 2024
5bacdcd
[Q20-15] reverting the code factor issue fix
pooja-thalur Sep 18, 2024
f28fc45
Merge branch 'develop' into story/Q20-15
pooja-thalur Dec 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions quadriga/.classpath
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,6 @@
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
<classpathentry kind="con" path="org.eclipse.jdt.junit.JUNIT_CONTAINER/5"/>
<classpathentry kind="output" path="target/classes"/>
</classpath>
18 changes: 14 additions & 4 deletions quadriga/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@
<admin.password>$2a$04$oQo44vqcDIFRoYKiAXoNheurzkwX9dcNmowvTX/hsWuBMwijqn44i</admin.password>

<db.driver>com.mysql.jdbc.Driver</db.driver>
<db.database.url>jdbc-url-to-db</db.database.url>
<db.user>db-user</db.user>
<db.password>db-password</db.password>
<db.database.url>jdbc:mysql://localhost:3306/quadriga</db.database.url>
<db.user></db.user>
<db.password></db.password>


<email.user></email.user>
<email.password></email.password>
Expand All @@ -41,10 +42,19 @@
<neo4j.url>http://neo4j:quadriga@localhost:7474/</neo4j.url>
<neo4j.database>quadriga</neo4j.database>

<citesphere.base.url></citesphere.base.url>
<quadriga.base.url></quadriga.base.url>
<quadriga.jobstatus.api></quadriga.jobstatus.api>
<quadriga.collectionpage.url></quadriga.collectionpage.url>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about these two (line 46 and 47). These are path from Quadriga itself, right? Just create a constant in the controller with that path and use the constant when you need it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?


<citesphere.base.url>https://diging-dev.asu.edu/citesphere-review/</citesphere.base.url>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not be changed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should have a dummy default value

<citesphere.client.id></citesphere.client.id>
<citesphere.client.secret></citesphere.client.secret>
jdamerow marked this conversation as resolved.
Show resolved Hide resolved

<citesphere.client.id></citesphere.client.id>
<citesphere.client.secret></citesphere.client.secret>


<conceptpower.base.url></conceptpower.base.url>
<log.level>info</log.level>

<tomcat.deploy.path></tomcat.deploy.path>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package edu.asu.diging.quadriga.api.v1;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

import edu.asu.diging.quadriga.core.model.jobs.Job;
import edu.asu.diging.quadriga.core.service.JobManager;

public class JobStatusController {

@Autowired
private JobManager jobManager;


@GetMapping(value = "/api/v1/job/{jobId}/status")
public ResponseEntity<Job> getJobStatus(@PathVariable String jobId) {
Job job = jobManager.get(jobId);
if(job == null){
return new ResponseEntity<>(null,HttpStatus.NOT_FOUND);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain there is a constructor that just takes an httpstatus.

}
return new ResponseEntity<>(job, HttpStatus.OK);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
package edu.asu.diging.quadriga.api.v1;

import java.util.ArrayList;

import java.util.List;

import org.bson.types.ObjectId;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;

import edu.asu.diging.quadriga.api.v1.model.PatternMapping;
import edu.asu.diging.quadriga.api.v1.model.PatternMappingList;
import edu.asu.diging.quadriga.core.model.EventGraph;
import edu.asu.diging.quadriga.core.model.MappedTripleGroup;
import edu.asu.diging.quadriga.core.model.jobs.Job;
import edu.asu.diging.quadriga.core.service.AsyncPatternProcessor;
import edu.asu.diging.quadriga.core.service.EventGraphService;
import edu.asu.diging.quadriga.core.service.JobManager;

@Controller
public class MapGraphToTripleController {

@Value("${quadriga_base_url}")
private String quadrigaBaseUri;

@Value("${quadriga_job_status_api}")
private String quadrigaJobStatusUri;

@Value("${quadriga_collection_page}")
private String quadrigaCollectionPageUri;

@Autowired
private AsyncPatternProcessor asyncPatternProcessor;

@Autowired
private EventGraphService eventGraphService;

@Autowired
private JobManager jobManager;

@PostMapping(value = "/api/v1/collection/{collectionId}/network/map")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should just be /api/v1/collection/{collectionId}/map

public ResponseEntity<List<JobPatternInfo>> mapPatternToTriples(@PathVariable String collectionId,
@RequestBody PatternMappingList patternMappingList) {

List<EventGraph> eventGraphs = eventGraphService.getEventGraphs(new ObjectId(collectionId));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these can be a lot of graphs. You don't want to load them all into memory. I don't think you want this done here.

if (eventGraphs == null) {
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the logic should rather be to check if the collection exists, not the if event graphs exists. it's fine (even if it doesn't make much sense) to try to run mappings on 0 event graphs. An invalid collection id, however, should return a not found.

}

List<JobPatternInfo> jobInfos = new ArrayList<>();

MappedTripleGroup mappedTripleGroup = new MappedTripleGroup();
mappedTripleGroup.set_id(new ObjectId());
mappedTripleGroup.setCollectionId(new ObjectId(collectionId));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should not be part of the controller, that's business logic.


String jobId = jobManager.createJob(collectionId, mappedTripleGroup.get_id().toString(), eventGraphs.size());

List<Thread> transformationThreads = new ArrayList<>();
for (PatternMapping pattern : patternMappingList.getPatternMappings()) {
Thread transformationThread = new Thread(() -> {
asyncPatternProcessor.processPattern(jobId, collectionId, pattern, eventGraphs);
JobPatternInfo jobInfo = new JobPatternInfo();
jobInfo.setJobId(jobId);
jobInfo.setTrack(quadrigaBaseUri + quadrigaJobStatusUri + jobId + "/status");
jobInfo.setExplore(quadrigaBaseUri + quadrigaCollectionPageUri + collectionId);
jobInfos.add(jobInfo);
});
transformationThreads.add(transformationThread);
transformationThread.start();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this above is all business logic and should not be done in the controller

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, you should create threads; use the async functionality of spring instead

for (Thread thread : transformationThreads) {
try {
thread.join();
} catch (InterruptedException e) {
System.out.println("Job thread was interrupted due to the exception:"+e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

???

}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is all not necessary when using Spring's async

return new ResponseEntity<>(jobInfos, HttpStatus.OK);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be just one job for a transformation request. There might be multiple threads started to work on the individual patterns, but there should only be one job.

}

@GetMapping(value = "/api/v1/job/{jobId}/status")
public ResponseEntity<Job> getJobStatus(@PathVariable String jobId) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be in it's own controller

return new ResponseEntity<>(jobManager.get(jobId), HttpStatus.OK);
}

class JobPatternInfo {
private String jobId;
private String track;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is 'track'?

private String explore;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is 'explore'?


public String getJobId() {
return jobId;
}
public void setJobId(String jobId) {
this.jobId = jobId;
}
public String getTrack() {
return track;
}
public void setTrack(String track) {
this.track = track;
}
public String getExplore() {
return explore;
}
public void setExplore(String explore) {
this.explore = explore;
}

}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package edu.asu.diging.quadriga.api.v1.model;

import java.util.List;
import java.util.Map;

public class PatternMapping {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs javadoc


private String mappedTripleGroupId;
jdamerow marked this conversation as resolved.
Show resolved Hide resolved
private Metadata metadata;
private Map<String, PatternNodeData> nodes;
private List<Edge> edges;

public String getMappedTripleGroupId() {
return mappedTripleGroupId;
}
public void setMappedTripleGroupId(String mappedTripleGroupId) {
this.mappedTripleGroupId = mappedTripleGroupId;
}
public Metadata getMetadata() {
return metadata;
}
public void setMetadata(Metadata metadata) {
this.metadata = metadata;
}
public Map<String, PatternNodeData> getNodes() {
return nodes;
}
public void setNodes(Map<String, PatternNodeData> nodes) {
this.nodes = nodes;
}
public List<Edge> getEdges() {
return edges;
}
public void setEdges(List<Edge> edges) {
this.edges = edges;
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package edu.asu.diging.quadriga.api.v1.model;

import java.util.List;

public class PatternMappingList {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this class for?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this class needed? Can't you just use a list?


private List<PatternMapping> patternMappings;

public List<PatternMapping> getPatternMappings() {
return patternMappings;
}

public void setPatternMappings(List<PatternMapping> patternMappings) {
this.patternMappings = patternMappings;
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package edu.asu.diging.quadriga.api.v1.model;

public class PatternNodeData {

private String type;
private String interpretation;
private String conceptType;

public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getInterpretation() {
return interpretation;
}
public void setInterpretation(String interpretation) {
this.interpretation = interpretation;
}
public String getConceptType() {
return conceptType;
}
public void setConceptType(String conceptType) {
this.conceptType = conceptType;
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ public PersistenceExceptionTranslationPostProcessor exceptionTranslation() {
Properties additionalProperties() {
Properties properties = new Properties();
properties.setProperty("hibernate.hbm2ddl.auto", "update");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
properties.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL8Dialect");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not be changed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

properties.setProperty("hibernate.show_sql", "true");
properties.setProperty("hibernate.id.new_generator_mappings", "true");

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,11 @@
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.context.support.ReloadableResourceBundleMessageSource;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;

@Configuration
@EnableAsync
@EnableCaching
@EnableWebMvc
@EnableAspectJAutoProxy
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package edu.asu.diging.quadriga.core.conceptpower;

import edu.asu.diging.quadriga.core.model.conceptpower.ConceptEntry;
import edu.asu.diging.quadriga.core.model.conceptpower.ConceptpowerResponse;

public interface ConceptpowerConnector {

/**
* Retrieves the concept for the given id from Conceptpower
* @param id is the concept id
* @return the retrieved concept
*/
ConceptEntry getConceptEntry(String id);

/**
* Retrieves the concept entries which are equal to the given interpretation
* @param uri URI of interpretation
* @return the retrieved results
*/
ConceptpowerResponse findConceptEqualTo(String uri);

}
Loading