Skip to content

Commit

Permalink
Release (#190)
Browse files Browse the repository at this point in the history 
* [ALS-5827] Enable search by CA id

- If a uuid does not match a query, instead search for common area UUID
- If we have a collision, I'm buying a lottery ticket

* [ALS-5787] Open Access StatViz is not showing "Other" (#180)

* Update key length handling in limitKeySize

The limitKeySize method in the VisualizationUtil class has been updated to ensure key uniqueness when keys exceed a certain length. A more robust method of shortening the keys has been implemented: if the key is longer than 45 characters, it is cut off and replaced with "..." and additional characters are appended to ensure uniqueness if needed. This helps limit key size while preserving uniqueness.

* Add VisualizationUtilTests and refine key shortening logic

Added a new test class, VisualizationUtilTests to validate the functionality of the VisualizationUtil class. Adjusted the 'limitKeySize' method to ensure key uniqueness when keys are shortened to a maximum length of 45 characters. If shortened keys are not unique, additional trailing characters are included until uniqueness is achieved.

* Add check on length before substring

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-resources/pic-sure-visualization-resource/src/test/java/edu/harvard/hms/dbmi/avillach/resource/visualization/service/VisualizationUtilTests.java

Co-authored-by: Luke Sikina <lucas.sikina@gmail.com>

* Fix import for modified unit test

* Refactor and add test cases for key size limiting function

Refactors existing code by simplifying key size limiting logic in VisualizationUtil and added several new unit tests to ensure its correct behavior with different input scenarios including long keys, empty maps, null maps, and uniqueness near middle.

* Update null handling in VisualizationUtil's limitKeySize

Changed the handling of null input in VisualizationUtil's limitKeySize from returning a new HashMap to throwing an IllegalArgumentException. Also, updated the corresponding test to check for this exception instead of comparing with an empty map.

---------

Co-authored-by: Luke Sikina <lucas.sikina@gmail.com>

* Refine the condition for empty return in DataProcessingService (#184)

Adjusted the condition under which an empty HashMap is returned in the DataProcessingService. Now, an empty HashMap will only be returned when both the maximum and minimum values in the data are 0, and the number of bins is also 0. This change takes into account situations where the data contains values but the min and max are both 0.

* ALS-5387: Remove stack from resources table (#186)

* [ALS-5422] Add persistence.xml to visualization resource (#161) (#162)
The visualization resource is failing to start due to an error injecting persistence unit into CDI managed bean. It is unable to find a persistence unit named ''.

* ALS-5387: Remove stack from resources table
* ALS-5387: More robust implementation of feature

---------

Co-authored-by: Gcolon021 <34667267+Gcolon021@users.noreply.github.com>
Co-authored-by: GeorgeC <gcolon021@gmail.com>

* [ALS-6222] Status endpoints now filters resources (#189)

The database contains both open and auth hpds even if it wasn't deployed. This means we attempt to check for a resource that doesn't exist. This results in our service always showing degraded.

---------

Co-authored-by: Luke Sikina <lucas.skina@gmail.com>
Co-authored-by: Luke Sikina <lucas.sikina@gmail.com>
Co-authored-by: ramari16 <ramari16@gmail.com>
  • Loading branch information
@Gcolon021 @Luke-Sikina @ramari16
4 people authored Apr 2, 2024
1 parent 32ab9dd commit 89a823d
Show file tree
Hide file tree
Showing 2 changed files with 173 additions and 144 deletions.
21 changes: 14 additions & 7 deletions pic-sure-api-war/src/main/java/edu/harvard/dbmi/avillach/PicSureWarInit.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ public class PicSureWarInit {
@Resource(mappedName = "java:global/token_introspection_token")
private String token_introspection_token;

//to be able to pre modified
@Resource(mappedName = "java:global/defaultApplicationUUID")
private String default_application_uuid;

// to be able to pre modified
public static final ObjectMapper objectMapper = new ObjectMapper();

// check the example from Apache HttpClient official website:
Expand All @@ -39,19 +42,23 @@ public class PicSureWarInit {
static {
HTTP_CLIENT_CONNECTION_MANAGER = new PoolingHttpClientConnectionManager();
HTTP_CLIENT_CONNECTION_MANAGER.setMaxTotal(100);
CLOSEABLE_HTTP_CLIENT = HttpClients
.custom()
.setConnectionManager(HTTP_CLIENT_CONNECTION_MANAGER)
.useSystemProperties()
.build();
CLOSEABLE_HTTP_CLIENT = HttpClients.custom().setConnectionManager(HTTP_CLIENT_CONNECTION_MANAGER).useSystemProperties().build();
}


public String getToken_introspection_url() {
return token_introspection_url;
}

public String getToken_introspection_token() {
return token_introspection_token;
}

/**
* This method is used to get the default application UUID. This value is either the open or auth hpds resource UUID.
*
* @return the default application UUID
*/
public String getDefaultApplicationUUID() {
return this.default_application_uuid;
}
}
296 changes: 159 additions & 137 deletions pic-sure-api-war/src/main/java/edu/harvard/dbmi/avillach/service/SystemService.java
View file
Original file line number Diff line number Diff line change
@@ -1,137 +1,159 @@
package edu.harvard.dbmi.avillach.service;
import static edu.harvard.dbmi.avillach.util.Utilities.buildHttpClientContext;

import java.io.IOException;
import java.util.List;

import javax.annotation.PostConstruct;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

import edu.harvard.dbmi.avillach.PicSureWarInit;
import edu.harvard.dbmi.avillach.data.entity.Resource;
import edu.harvard.dbmi.avillach.data.repository.ResourceRepository;
import edu.harvard.dbmi.avillach.domain.GeneralQueryRequest;
import edu.harvard.dbmi.avillach.domain.ResourceInfo;
import edu.harvard.dbmi.avillach.util.exception.ApplicationException;

@Path("/system")
public class SystemService {
static int max_test_frequency = 60000;

static final String RUNNING = "RUNNING";

static final String ONE_OR_MORE_COMPONENTS_DEGRADED = "ONE OR MORE COMPONENTS DEGRADED";

Logger logger = LoggerFactory.getLogger(SystemService.class);

@Inject
PicSureWarInit picSureWarInit;

String lastStatus = "UNTESTED";
long lastStatusCheck = 0l;

@Inject
ResourceRepository resourceRepo;

String token_introspection_url;
String token_introspection_token;

@PostConstruct
public void init() {
token_introspection_url = picSureWarInit.getToken_introspection_url();
token_introspection_token = picSureWarInit.getToken_introspection_token();
if(token_introspection_url == null || token_introspection_token == null) {
throw new RuntimeException(
"token_introspection_url and token_introspection_token not configured");
}
}

@GET
@Path("/status")
@Produces("text/plain")
public String status() {
// Because there is no auth on this service we limit actually performing the checking to 1 per minute to avoid DOS scenarios.
long timeOfRequest = System.currentTimeMillis();
if(timeOfRequest-lastStatusCheck < max_test_frequency) {
return lastStatus;
}else {
lastStatusCheck = timeOfRequest;
try{
List<Resource> resourcesToTest = resourceRepo.list();
if( resourcesToTest != null && // This proves the MySQL database is serving queries
!resourcesToTest.isEmpty() && // This proves at least one resources is configured
testPSAMAResponds() && // This proves we can perform token introspection
testResourcesRespond(resourcesToTest) ){ // This proves all resources are at least serving info requests.
lastStatus = RUNNING;
return lastStatus;
}else {
lastStatus = ONE_OR_MORE_COMPONENTS_DEGRADED;
}
}catch(Exception e) {
e.printStackTrace();
lastStatus = ONE_OR_MORE_COMPONENTS_DEGRADED;
}
return lastStatus;
}
}

private boolean testPSAMAResponds() throws UnsupportedOperationException, IOException {
CloseableHttpClient client = PicSureWarInit.CLOSEABLE_HTTP_CLIENT;
ObjectMapper json = PicSureWarInit.objectMapper;

HttpPost post = new HttpPost(token_introspection_url);
post.setEntity(new StringEntity("{}"));
post.setHeader("Content-Type", "application/json");
//Authorize into the token introspection endpoint
post.setHeader("Authorization", "Bearer " + token_introspection_token);
CloseableHttpResponse response = null;
try {
response = client.execute(post, buildHttpClientContext());
if (response.getStatusLine().getStatusCode() != 200){
logger.error("callTokenIntroEndpoint() error back from token intro host server ["
+ token_introspection_url + "]: " + EntityUtils.toString(response.getEntity()));
throw new ApplicationException("Token Introspection host server return " + response.getStatusLine().getStatusCode() +
". Please see the log");
}
JsonNode responseContent = json.readTree(response.getEntity().getContent());
if (!responseContent.get("active").asBoolean()){
// This is actually the expected response as we did not send a token in the token_introspection_request.
return true;
}

return true;
} finally {
try {
if (response != null)
response.close();
} catch (IOException ex) {
logger.error("callTokenIntroEndpoint() IOExcpetion when closing http response: " + ex.getMessage());
}
}
}

private boolean testResourcesRespond(List<Resource> resourcesToTest) {
for(Resource resource : resourcesToTest) {
ResourceInfo info = new ResourceWebClient().info(resource.getResourceRSPath(), new GeneralQueryRequest());
if(info==null) {
return false;
}
}
return true;
}
}

package edu.harvard.dbmi.avillach.service;

import static edu.harvard.dbmi.avillach.util.Utilities.buildHttpClientContext;

import java.io.IOException;
import java.util.List;
import java.util.UUID;

import javax.annotation.PostConstruct;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

import edu.harvard.dbmi.avillach.PicSureWarInit;
import edu.harvard.dbmi.avillach.data.entity.Resource;
import edu.harvard.dbmi.avillach.data.repository.ResourceRepository;
import edu.harvard.dbmi.avillach.domain.GeneralQueryRequest;
import edu.harvard.dbmi.avillach.domain.ResourceInfo;
import edu.harvard.dbmi.avillach.util.exception.ApplicationException;

@Path("/system")
public class SystemService {
static int max_test_frequency = 60000;

static final String RUNNING = "RUNNING";

static final String ONE_OR_MORE_COMPONENTS_DEGRADED = "ONE OR MORE COMPONENTS DEGRADED";

Logger logger = LoggerFactory.getLogger(SystemService.class);

@Inject
PicSureWarInit picSureWarInit;

String lastStatus = "UNTESTED";
long lastStatusCheck = 0l;

@Inject
ResourceRepository resourceRepo;

String token_introspection_url;
String token_introspection_token;

String defaultApplicationUUID;

@PostConstruct
public void init() {
token_introspection_url = picSureWarInit.getToken_introspection_url();
token_introspection_token = picSureWarInit.getToken_introspection_token();
defaultApplicationUUID = picSureWarInit.getDefaultApplicationUUID();
if (token_introspection_url == null || token_introspection_token == null) {
throw new RuntimeException("token_introspection_url and token_introspection_token not configured");
}
}

@GET
@Path("/status")
@Produces("text/plain")
public String status() {
// Because there is no auth on this service we limit actually performing the checking to 1 per minute to avoid DOS scenarios.
long timeOfRequest = System.currentTimeMillis();
if (timeOfRequest - lastStatusCheck < max_test_frequency) {
return lastStatus;
} else {
lastStatusCheck = timeOfRequest;
try {
List<Resource> resourcesToTest = resourceRepo.list();
if (resourcesToTest == null || resourcesToTest.isEmpty()) {
lastStatus = ONE_OR_MORE_COMPONENTS_DEGRADED;
return lastStatus;
}

// convert the default application uuid to an uuid object
UUID defaultApplicationUUID = UUID.fromString(this.defaultApplicationUUID);

// We need to remove open or auth HPDS from the resource list depending on the environment deployed.
// This because both are included in the database, but only one is actually deployed.
// if the name contains hpds and is not the default application uuid, remove it.
resourcesToTest.removeIf(
resource -> resource.getName().toLowerCase().contains("hpds") && !resource.getUuid().equals(defaultApplicationUUID)
);

// This proves the MySQL database is serving queries
// This proves at least one resources is configured
// This proves we can perform token introspection
if (testPSAMAResponds() && testResourcesRespond(resourcesToTest)) { // This proves all resources are at least serving info
// requests.
lastStatus = RUNNING;
return lastStatus;
} else {
lastStatus = ONE_OR_MORE_COMPONENTS_DEGRADED;
}
} catch (Exception e) {
e.printStackTrace();
lastStatus = ONE_OR_MORE_COMPONENTS_DEGRADED;
}
return lastStatus;
}
}

private boolean testPSAMAResponds() throws UnsupportedOperationException, IOException {
CloseableHttpClient client = PicSureWarInit.CLOSEABLE_HTTP_CLIENT;
ObjectMapper json = PicSureWarInit.objectMapper;

HttpPost post = new HttpPost(token_introspection_url);
post.setEntity(new StringEntity("{}"));
post.setHeader("Content-Type", "application/json");
// Authorize into the token introspection endpoint
post.setHeader("Authorization", "Bearer " + token_introspection_token);
CloseableHttpResponse response = null;
try {
response = client.execute(post, buildHttpClientContext());
if (response.getStatusLine().getStatusCode() != 200) {
logger.error(
"callTokenIntroEndpoint() error back from token intro host server [" + token_introspection_url + "]: "
+ EntityUtils.toString(response.getEntity())
);
throw new ApplicationException(
"Token Introspection host server return " + response.getStatusLine().getStatusCode() + ". Please see the log"
);
}
JsonNode responseContent = json.readTree(response.getEntity().getContent());
if (!responseContent.get("active").asBoolean()) {
// This is actually the expected response as we did not send a token in the token_introspection_request.
return true;
}

return true;
} finally {
try {
if (response != null) response.close();
} catch (IOException ex) {
logger.error("callTokenIntroEndpoint() IOExcpetion when closing http response: " + ex.getMessage());
}
}
}

private boolean testResourcesRespond(List<Resource> resourcesToTest) {
for (Resource resource : resourcesToTest) {
ResourceInfo info = new ResourceWebClient().info(resource.getResourceRSPath(), new GeneralQueryRequest());
if (info == null) {
return false;
}
}
return true;
}
}

0 comments on commit 89a823d

Please sign in to comment.