Skip to content

Commit

Permalink
handler docstring change
Browse files Browse the repository at this point in the history
  • Loading branch information
saileshd1402 committed Oct 13, 2023
1 parent 2f35bf5 commit 1aa6c66
Showing 1 changed file with 1 addition and 17 deletions.
18 changes: 1 addition & 17 deletions llm/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,6 @@ class LLMHandler(BaseHandler, ABC):
Attributes:
initialized (bool): Flag indicating that Torchserve is initialized.
request_list (dict):
request_ids (dict):
request_type (dict):
device (torch.device): The device that is being used for generation.
map_location(str): The type of device where the model is
Expand All @@ -77,13 +71,12 @@ class LLMHandler(BaseHandler, ABC):
the provided model name and model files present in MAR file.
preprocess(text: str) -> Tensor:
This method tookenizes input text using the associated tokenizer.
This method tokenizes input text using the associated tokenizer.
Args:
text (str): The input text to be tokenized.
Returns:
Tensor: Tokenized input data
inference(data: Tensor) -> list(str):
This method reads the generation parameters set as environment vairables
and uses the preprocessed tokens and generation parameters to generate a
Expand All @@ -99,15 +92,6 @@ class LLMHandler(BaseHandler, ABC):
data (list(str)): A list containing the output text of model generation.
Returns:
list(str): A list containing model's generated output.
_batch_to_json():
_to_json():
get_env_value(str) -> float:
This method reads the inputed environment variable and converts it to float
and returns it. This is used for reading model generation parameters.
Args:
env_var (str): Environment variable to be read.
Returns:
float: Value of the enviroment variable read.
"""

def __init__(self):
Expand Down

0 comments on commit 1aa6c66

Please sign in to comment.