Upcoming versions (to be updated to) - Announce/Warn the users before doing major changes like in v3.5.0

[SINAPSA-IC]

It is only sensible that the users be announced or warned before they update to a version where major, critical changes/functionality is implemented that Could impact negatively on user experience (UX).
This way, if said changes somehow disrupt the workflow that Most users are already accustomed with - primarily and most importantly regarding the ease-of-use - they would know so that they may choose to skip such updates, and/or wait until Issues/Bugs with those changes are sorted out.

Reason:
In v3.5.0 the Prompt Template format changed to Jinja, a major change (as said on the Welcome screen "Chat (Prompt, that is - the user has to guess what a "chat template" is) templates have been completely overhauled" - which upended the use of a host of models whose Prompt Template is not understood anymore by the program. As a result, the users are effectively forced to write the new templates by hand; what's more (or less), the PTs of some models (like [INST] %1 [/INST] are not documented on 2024.12.09 when v3.5.0 came out, however in the documentation there's the term "(control tokens) such as" - which neither explains to the Common User what a control token is, nor which are the rest of control tokens within the set that contains the particular control tokens referred by "such as". What the Common User wants is 1 the program to just work, 2 not to fiddle themselves with functionality that Should be working out-of-the-box, 3 not be sent to read a litany of documentation (and an incomplete/unhelpful one at that) - when even this one message poses comprehension problems because of its length only (length comes before contents), 4 not act as testers.

Idea/Easy solution:
1.
The choice to Update should be given to the user

• either alongside an announcement or warning placed visibly and explicitly, saying what could happen after the update, right at the start of the program. As of yet, the "Üpdate" dialog appears before showing the main view, but no warning whatsoever ref. major changes appears alongside the button, nor afterwards in the download windows for the updated-files.
• or move the Check for Updates button from the bottom-right corner of a Settings page, to a more accessible place on the UI

Or, create an element on the UI where upcoming Major changes are announced, like "we are working on something major for the next version, so you may want to wait and see if it works 100%, and only then update ". This control, like a Label, would only appear when such a message is prepared by the developers.

As for the Need to implement the Jinja format in the UI - when most users are using the GUI, instead of doing low-level coding where they would have to write by hand such a PT - it is a functionality that took time and resources for a functionality that the user base at large probably did not 1need=want, 2 know it could exist, 3 know that has become absolutely necessary for them operating with the program because they were not forewarned about it. To work on such functionality deserves a good motivation that the Users (too) (not the developers and the few users familiar with the inner workings of LLMs, such as PTs) could both understand and accept.

[aureagle]

Upgrading to 3.5.1 made the CPU work very hard for me, like 90% while the model was loaded on the GPU, so it seems that my GPU is not being used at full potential like it used to be in the past version, The text generation was extremely degraded 3.5.1. I'm downgrading now, hope things get back to previous (controllable) bugs!

[SINAPSA-IC]

@ aureagle, I've also seen this, but it may be related to the setting in Windows (where Windows is the OS, ofc) where the Performance is optimized for applications or for background services, especially in Win11 that takes it seriously. The GPU load in either case may be tangential...

[UserHub1973]

The Current update 3.5.1 broke all the models I use daily. Across the board. Not compatible any longer.

Yes indeed, their curated models work; but those were never the reason I used the software.

Side loading was.

[UserHub1973]

So out of 12 models that are regularly used by me.

None of them work as of the last update. They all worked wonderfully under 3.4.2 and all are broke under 3.5.1.

Is every non Nomic Model being rendered useless and inoperable, going to be addressed as a FLAW or is it a new feature that I am to be happy and joyous over at all times?

Basically, is there a plan to go back to some kind of 'jinja' based 'default prompt' that just works with any side loaded models? Lawyer talk and nanny gas lighting aside the expected real-world behavior was side loaded models just worked once put into the model folder. Very compatible with many many models of any quant we wanted from 4 to 16 any quant you wanted almost any model. Thousands in fact.

That was the functionality up until 3.5 AFTER ALL. Truth be told. 99% of all Hugging face models just worked out of the box and with very little or NO fiddling, upside, will all those THOUSAND OF LLM models on HUGGINGFACE that made your product popular because they side loaded EASILY, have to stop advertising as working with your product?

TheBloke Models are not compatible with software all the sudden?

TheBloke/OpenHermes-2.5-Mistral-7B-GGUF
Ran script ... Won't just find and install the correct template?

Nope. This is a lot of lost productivity.

The Bloke and folks like him ... MADE YOU ...

Those 'side loaded' models are why many folks use you. We ourselves, only use this because it gave us the EASE and FREEDOM to use any model WE CHOSE on our machine under OUR TERMS. 99/100 AI models in 5 or 6 Archs used to work perfectly with the default out of the box.

If that goes: Your software gets uninstalled. I don't use your curated ones at all daily so them working or not matters not to me in the least. My freedom to use the model I want like I could last week, does.

[UserHub1973]

I see this place is still gas lighting folks with 'less is more' and you 'never really had all those models to use' at any rate.

Pure Legalist lies ...

[UserHub1973]

List of broken models:

https://huggingface.co/TheBloke 99% of these worked prior to this 'update' -- ANYTHING ELSE IS SPIN.

And another list of broken models:

https://huggingface.co/cognitivecomputations -- same situtation ... broke by design.

And these ... Of them only the 70bs would not load and inference ... IMHO.

Now NONE OF THEM WORK OUT of the box ... And we are supposed to pretend that they NEVER WORKED.

That is what the discord seems to be preaching: Those models were always 'ify' ... Uh huh ... out of several Hundred I have tried in 1 year; two failed to load and inference! TWO out of at least 200 ... I saved the better ones to a disk ... I have 97 models I saved!
Try to gaslight me.

OKAY?

Stop with the gaslighting ... It is not a step foward for normal users to have less choices with more difficulty.

Learning to CODE IS NOT NEEDED to side load ... Any notion it should be a SELF SERVING FIB. That is Gaslighting.

Your downloader in your OWN SOFTWARE will download ALL THESE ... BROKE ... and unworking.

That was a CHOICE you made. And not a good one.

Oh you broke our beloved models to give us YOUR MODELS and to usher in 'agents' executing code.

WHY would I want YOUR CLOSED SOURCE MODELS having any agency on my machine at all?

Fact is you don't want us to be able to run any other models but Nomics -- that leads to the QUESTION: WHY?

Control. That is the ONLY LOGICAL ANSWER. Which also explains the gaslighting and attempts to blame\shame the user base for rejecting your 'updates'.

[Tothefight]

I think the possibility of easier tool use in GPT4ALL is kinda worth the headache, Of coarse, at first, I was furious when I thought my setups were messed up, but there should be basic Jinja templates to make the older models work. I was pretty pissed because all my Qwen models wouldn't work but luckily I uploaded a similar one from their list and used that template on the others and it's faster, seems to build up to its optimal speed rather than slowly decrease in Tokens per second but I'm looking forward to the tool calling.

Most people would say, "It's free so you can't really complain." or "It's only a few people working on the project." I mean if it is open source there should be a better update system if they are expecting people to use this in their day to day lives. GPT4ALL is one of the few Open Source UIs letting normal people use AI instead of a large corporate project where the people pay just so the AI can essentially use them... So I can't really express how much gratitude I have towards the developers and I'm a Scribble-Chimp(Writer) rather than a Code-Monkey(Coder) but I have been spending some time with my over 100 models getting them working so if you guys need any specific Jinja templates let me know.

[UserHub1973]

I think the possibility of easier tool use in GPT4ALL is kinda worth the headache, Of coarse, at first, I was furious when I thought my setups were messed up, but there should be basic Jinja templates to make the older models work. I was pretty pissed because all my Qwen models wouldn't work but luckily I uploaded a similar one from their list and used that template on the others and it's faster, seems to build up to its optimal speed rather than slowly decrease in Tokens per second but I'm looking forward to the tool calling.

Most people would say, "It's free so you can't really complain." or "It's only a few people working on the project." I mean if it is open source there should be a better update system if they are expecting people to use this in their day to day lives. GPT4ALL is one of the few Open Source UIs letting normal people use AI instead of a large corporate project where the people pay just so the AI can essentially use them... So I can't really express how much gratitude I have towards the developers and I'm a Scribble-Chimp(Writer) rather than a Code-Monkey(Coder) but I have been spending some time with my over 100 models getting them working so if you guys need any specific Jinja templates let me know.

Sorry man but adding MORE Steps to get AI running out of the BOX is not easier in any realworld situation. No I won't jump on one foot or beg. I will just use other SOFTWARE that values its users experience better.

AnythingLLM will RAG -- several dozens gig of docs; more kind of docs; and you can import any model that worked in 3.4.2 just like you own the place.

LMStupio inferneces like champ any model ... every time. Never had a failure but have had to get the prompt right ... Still easier than learning an undocumented in house coding dialect.

[UserHub1973]

I need something like this to work:

{%- if messages[0]["role"] == "system" %}
{%- set system_message = messages[0]["content"] %}
{%- set loop_messages = messages[1:] %}
{%- else %}
{%- set loop_messages = messages %}
{%- endif %}
{%- if not tools is defined %}
{%- set tools = none %}
{%- endif %}
{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}

{%- for message in loop_messages | rejectattr("role", "equalto", "tool") | rejectattr("role", "equalto", "tool_results") | selectattr("tool_calls", "undefined") %}
{%- if (message["role"] == "user") != (loop.index0 % 2 == 0) %}
{{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
{%- endif %}
{%- endfor %}

{{- bos_token }}
{%- for message in loop_messages %}
{%- if message["role"] == "user" %}
{%- if tools is not none and (message == user_messages[-1]) %}
{{- "[AVAILABLE_TOOLS] [" }}
{%- for tool in tools %}
{%- set tool = tool.function %}
{{- '{"type": "function", "function": {' }}
{%- for key, val in tool.items() if key != "return" %}
{%- if val is string %}
{{- '"' + key + '": "' + val + '"' }}
{%- else %}
{{- '"' + key + '": ' + val|tojson }}
{%- endif %}
{%- if not loop.last %}
{{- ", " }}
{%- endif %}
{%- endfor %}
{{- "}}" }}
{%- if not loop.last %}
{{- ", " }}
{%- else %}
{{- "]" }}
{%- endif %}
{%- endfor %}
{{- "[/AVAILABLE_TOOLS]" }}
{%- endif %}
{%- if loop.last and system_message is defined %}
{{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
{%- else %}
{{- "[INST] " + message["content"] + "[/INST]" }}
{%- endif %}
{%- elif message["role"] == "tool_calls" or message.tool_calls is defined %}
{%- if message.tool_calls is defined %}
{%- set tool_calls = message.tool_calls %}
{%- else %}
{%- set tool_calls = message.content %}
{%- endif %}
{{- "[TOOL_CALLS] [" }}
{%- for tool_call in tool_calls %}
{%- set out = tool_call.function|tojson %}
{{- out[:-1] }}
{%- if not tool_call.id is defined or tool_call.id|length < 9 %}
{{- raise_exception("Tool call IDs should be alphanumeric strings with length >= 9! (1)" + tool_call.id) }}
{%- endif %}
{{- ', "id": "' + tool_call.id[-9:] + '"}' }}
{%- if not loop.last %}
{{- ", " }}
{%- else %}
{{- "]" + eos_token }}
{%- endif %}
{%- endfor %}
{%- elif message["role"] == "assistant" %}
{{- " " + message["content"] + eos_token }}
{%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
{%- if message.content is defined and message.content.content is defined %}
{%- set content = message.content.content %}
{%- else %}
{%- set content = message.content %}
{%- endif %}
{{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
{%- if not message.tool_call_id is defined or message.tool_call_id|length < 9 %}
{{- raise_exception("Tool call IDs should be alphanumeric strings with length >= 9! (2)" + message.tool_call_id) }}
{%- endif %}
{{- '"call_id": "' + message.tool_call_id[-9:] + '"}[/TOOL_RESULTS]' }}
{%- else %}
{{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
{%- endif %}
{%- endfor %}

That is the ONE from COLAB but it NO go under GTP4ALL .... Mystery of mysteries.

[UserHub1973]

This is MISTRAL INSTRUCT one of their flagship models ... THEIR VERY MODEL!

Prompt: What the F&&&?
Response: Error: Failed to parse chat template: 1:1: error: Unexpected exception occurred during template processing. Exception: Jinja template error: After the optional system message, conversation roles must alternate user/assistant/user/assistant/...

Not read .... says it is right ... but NOT!

And when you do reset it -- and magically it works ON the second GO!
It has 'limited' context.

Like it is not using its context window. AT ALL.

{%- if messages[0]['role'] == 'system' %}
{%- set system_message = messages[0]['content'] %}
{%- set loop_start = 1 %}
{%- else %}
{%- set loop_start = 0 %}
{%- endif %}
{%- for message in messages %}
{%- if loop.index0 >= loop_start %}
{%- if (message['role'] == 'user') != ((loop.index0 - loop_start) % 2 == 0) %}
{{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}
{%- endif %}
{%- if message['role'] == 'user' %}
{%- if loop.first and system_message is defined %}
{{- ' [INST] ' + system_message + '\n\n' + message['content'] + ' [/INST]' }}
{%- else %}
{{- ' [INST] ' + message['content'] + ' [/INST]' }}
{%- endif %}
{%- elif message['role'] == 'assistant' %}
{{- ' ' + message['content'] + eos_token }}
{%- else %}
{{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}
{%- endif %}
{%- endif %}
{%- endfor %}

[Tothefight]

Are you using your tilda?

[UserHub1973]

They are going to ban me and make it harder for the rest of you to complain about POOR DESIGN CHOICES.

AnyLLM it is ...

[UserHub1973]

Four updates later and STILL NOT GENERIC TEMPLATE -- which they have -- they had to have had to do the hack coding -- and NO functional DOCUMENTATION on how to get side loaded models loaded.

[Tothefight]

You go get yourself a GGUF file and place it in your folder, try intelligentestate's huggingface they have templates on their models

[UserHub1973]

If you need A to make A ... and don't have A ... Well ...

The best way to create a chat template (A) is to start by using an existing one (A) as a reference.

That is really helpful.

<<<it may mention the name of a well-known standard template, such as ChatML, Alpaca, Vicuna. GPT4All does not yet include presets for these templates, so they will have to be found in other models or taken from the community.

So you can find them but not here ... Cuz your issue ...

[scrypt-kitty]

I was able to easily downgrade to v3.4.2 and keep my LocalDocs, models, etc.

The idea of looking through Huggingface vs GPT4All supported models was nice, but not at the current cost of the Jinja template change. I spent a couple days this week trying to get models I had been using before to work again to no avail.

Currently I am running both 3.4.2 and latest on my machine, though not at the same time.