ChatMl template with execution time

[robang74]

Hi, I am trying to have the execution time displayed after a prompt answered.

This is the ChatML template that I found

{%- for message in messages %}
    {{- '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n' }}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>' + 'assistant\n' }}
{%- endif %}

This the part that I have added at the end of the above code

{%- if execution_time is defined %}
    {{- '<|im_start|>metadata\nDebug: Execution Time Value = ' + execution_time|string + '<|im_end|>\n' }}
    {%- set seconds = execution_time|int %}
    {%- set milliseconds = ((execution_time - seconds) * 1000)|round|int %}
    {{- '<|im_start|>metadata\nExecution Time: ' + seconds|string + 's ' + milliseconds|string + 'ms<|im_end|>\n' }}
{%- else %}
    {{- '<|im_start|>metadata\nDebug: execution_time is NOT defined<|im_end|>\n' }}
{%- endif %}

Unfortunately, these data are not show in the gpt4all GUI and asking for the metadata at the prompt, it shows some included execution time which is usually 0.0001 sec or something unlikely real.

I do not think I am the first one that wishing to access some more information about the execution time but I did not find anything. Probably who are interested in this kind of data uses the CLI, instead.

Just wondering, R-