Feedback on SBI version 3.0-rc3

[ved-rivos]

SSE:

1. The section 17.5 does not address managing the ELP state
   introduced by the Zicfilp extension.
2. sbi_sse_complete - The description is missing the handling of the
   one-shot events which would move to REGISTERED state instead.
3. sbi_sse_inject - Add description of the behavior expected when
   a SSE event is injected from within an SSE handler.
4. Why is the event mask state not included in the STATUS attribute?

FWFT:

1. POINTER_MASKING_PMLEN. Allowing the enable of PM with
   length greater than request i.e. PMLEN > N will have
   unexpected consequences. For instance, a OS using Sv57
   may request PMLEN=7. If SBI picked PMLEN > 7 i.e. 15
   then it will lead to unexpected behavior in the OS.

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?
2. "notification indication" is redundant. Suggest to use
   either notification or indication.
3. sbi_mpxy_send_message_with_response
   "After sending the message, this function waits for the
   message response from the SBI implementation." This
   statement would want to be reworded to state that the
   function waits for "a response from the channel".
4. sbi_mpxy_send_message_with_response
   What is the handling for the case where the response
   data length exceeds the size of the shared memory.
   Is there an attribute - similar to MSG_DATA_MAX_LEN -
   required for enumerating the maximum response data
   length to allow the supervisor to size the shared memory
   to be the maximum of the two?

[atishp04]

Thanks for the feedback. We discussed these feedback during yesterday's PRS meeting.

SSE:

1. The section 17.5 does not address managing the ELP state
   introduced by the Zicfilp extension.

Yes. We will add that.

2. sbi_sse_complete - The description is missing the handling of the
   one-shot events which would move to REGISTERED state instead.

Will update.

3. sbi_sse_inject - Add description of the behavior expected when
   a SSE event is injected from within an SSE handler.

Sure.

4. Why is the event mask state not included in the STATUS attribute?

The event masking at a hart level instead of an event. So it's not a property of the event.
sbi_sse_hart_mask/unmask mask all the events on the calling hart.

FWFT:

1. POINTER_MASKING_PMLEN. Allowing the enable of PM with
   length greater than request i.e. PMLEN > N will have
   unexpected consequences. For instance, a OS using Sv57
   may request PMLEN=7. If SBI picked PMLEN > 7 i.e. 15
   then it will lead to unexpected behavior in the OS.

Yes. It is a valid issue. Discussion ongoing for potential solution.

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?

There were mixed opinions about this in the meeting.
While @pathakraul suggested to limit the API to messages without expecting a response,
others suggested that the original intent was.

The response retrieval mechanism is specific to the message protocol.
Since MPXY is just an abstract interface, it just allows blocking/unblocking
mechanisms to send a response. If the message protocol defines posted
request, it must provide appropriate additional mechanism to retrieve that
response.

For example: It can define another message type that can be send
using send_message_with_response to retrieve the response to a previously
sent msgID. This message type should have the ability to track previous msgID
for which it is seeking response. A message protocol may also not support posted request at all. In that case,
sbi_mpxy_send_message_without_response will be used only for messages
that doesn't expect response for that specific message protocol.
Currently, RPMI doesn't have any posted requests. So we don't have any examples to support the use case.
The supervisor software can either poll or wait for SSE/MSI notification to retrieve the response.

As per our offline discussion, it may not be prudent to have a place holder API without a valid use case in hand.
There are other three options suggested

1. Limit it to posted messages for which no response is expected
2. Correctly define response retrieval and not as a hack on top of send with response
3. Drop the send without response till a use case arises

I will start a thread on the mailing list for a broader audience engagement.
We definitely need to add more clarification for this API either way if it stays.

2. "notification indication" is redundant. Suggest to use
   either notification or indication.

Sure.

3. sbi_mpxy_send_message_with_response
   "After sending the message, this function waits for the
   message response from the SBI implementation." This
   statement would want to be reworded to state that the
   function waits for "a response from the channel".

Sure.

4. sbi_mpxy_send_message_with_response
   What is the handling for the case where the response
   data length exceeds the size of the shared memory.
   Is there an attribute - similar to MSG_DATA_MAX_LEN -
   required for enumerating the maximum response data
   length to allow the supervisor to size the shared memory
   to be the maximum of the two?

We will clarify MSG_DATA_MAX_LEN description to indicate that
it is meant for both message and response.

In addition to that, it was proposed during the meeting to define
an SBI function to return maximum of shared memory of all channels.
That would allow Supervisor to guess the shared memory required while
setting up the shared memory.
The SBI implementation is already aware of the shared memory size
requirement of each channel anyways.

[avpatel]

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?

There were mixed opinions about this in the meeting. While @pathakraul suggested to limit the API to messages without expecting a response, others suggested that the original intent was.

@atishp04 There were no mixed opinions in the PRS TG meeting. AFAIK, both myself and @pathakraul were suggesting the same thing that it is the responsibility of the message protocol to define appropriate mechanism to track posted messages.

The response retrieval mechanism is specific to the message protocol. Since MPXY is just an abstract interface, it just allows blocking/unblocking mechanisms to send a response. If the message protocol defines posted request, it must provide appropriate additional mechanism to retrieve that response.

For example: It can define another message type that can be send using send_message_with_response to retrieve the response to a previously sent msgID. This message type should have the ability to track previous msgID for which it is seeking response. A message protocol may also not support posted request at all. In that case, sbi_mpxy_send_message_without_response will be used only for messages that doesn't expect response for that specific message protocol. Currently, RPMI doesn't have any posted requests. So we don't have any examples to support the use case. The supervisor software can either poll or wait for SSE/MSI notification to retrieve the response.

As per our offline discussion, it may not be prudent to have a place holder API without a valid use case in hand. There are other three options suggested

1. Limit it to posted messages for which no response is expected
2. Correctly define response retrieval and not as a hack on top of send with response
3. Drop the send without response till a use case arises

The Option 1 is certainely better because the API is generic and abstract enough to allow message protocol specification define its own mechanism such as notification or another message to track the status or response for a posted message.

Regards,
Anup

[avpatel]

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?

There were mixed opinions about this in the meeting. While @pathakraul suggested to limit the API to messages without expecting a response, others suggested that the original intent was.

@atishp04 There were no mixed opinions in the PRS TG meeting. AFAIK, both myself and @pathakraul were suggesting the same thing that it is the responsibility of the message protocol to define appropriate mechanism to track posted messages.

The response retrieval mechanism is specific to the message protocol. Since MPXY is just an abstract interface, it just allows blocking/unblocking mechanisms to send a response. If the message protocol defines posted request, it must provide appropriate additional mechanism to retrieve that response.
For example: It can define another message type that can be send using send_message_with_response to retrieve the response to a previously sent msgID. This message type should have the ability to track previous msgID for which it is seeking response. A message protocol may also not support posted request at all. In that case, sbi_mpxy_send_message_without_response will be used only for messages that doesn't expect response for that specific message protocol. Currently, RPMI doesn't have any posted requests. So we don't have any examples to support the use case. The supervisor software can either poll or wait for SSE/MSI notification to retrieve the response.
As per our offline discussion, it may not be prudent to have a place holder API without a valid use case in hand. There are other three options suggested

1. Limit it to posted messages for which no response is expected
2. Correctly define response retrieval and not as a hack on top of send with response
3. Drop the send without response till a use case arises

The Option 1 is certainely better because the API is generic and abstract enough to allow message protocol specification to define its own mechanism such as notification or another message to track the status or response for a posted message.

How about a 4th Option where we only have single sbi_mpxy_send_message() call with an extra flags parameter where the bit[0] of flags parameter tells SBI implementation whether it is posted message or not ??

[clementleger]

@ved-rivos , the SSE spec does not describe which value is used for the SSTATUS.SDT bit when entering the SSE handler. In order to continue mimicking an interrupt (which is done currently), we should mandate the following:

• Upon event injection, interrupted_flags[5:5] = SSTATUS.SDT and SSTATUS.SDT = 1.
• When resuming, set SSTATUS.SDT = interrupted_flags[5:5]

A (very unlikely but still)) possible double trap with SSE scenario is the following:

• Trap happens
• Early trap handling is interrupt by an SSE event
• SSE event handler is called and start to save SCAUSE/STVAL/etc.
  • If a trap happens before saving these registers, it would override SCAUSE/STVAL content, leading to corrupted value when resuming execution (SBI does not save SCAUSE/STVAL).

[atishp04]

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?

There were mixed opinions about this in the meeting. While @pathakraul suggested to limit the API to messages without expecting a response, others suggested that the original intent was.

@atishp04 There were no mixed opinions in the PRS TG meeting. AFAIK, both myself and @pathakraul were suggesting the same thing that it is the responsibility of the message protocol to define appropriate mechanism to track posted messages.

I was referring to rahul's comment on the mailing list.
https://lists.riscv.org/g/tech-prs/message/1243

The response retrieval mechanism is specific to the message protocol. Since MPXY is just an abstract interface, it just allows blocking/unblocking mechanisms to send a response. If the message protocol defines posted request, it must provide appropriate additional mechanism to retrieve that response.
For example: It can define another message type that can be send using send_message_with_response to retrieve the response to a previously sent msgID. This message type should have the ability to track previous msgID for which it is seeking response. A message protocol may also not support posted request at all. In that case, sbi_mpxy_send_message_without_response will be used only for messages that doesn't expect response for that specific message protocol. Currently, RPMI doesn't have any posted requests. So we don't have any examples to support the use case. The supervisor software can either poll or wait for SSE/MSI notification to retrieve the response.
As per our offline discussion, it may not be prudent to have a place holder API without a valid use case in hand. There are other three options suggested

1. Limit it to posted messages for which no response is expected
2. Correctly define response retrieval and not as a hack on top of send with response
3. Drop the send without response till a use case arises

The Option 1 is certainely better because the API is generic and abstract enough to allow message protocol specification define its own mechanism such as notification or another message to track the status or response for a posted message.

Regards, Anup

[atishp04]

MPXY:

1. sbi_mpxy_send_message_without_response - this function
   states that it does not wait for a response. However,
   there does not seem to be a way to retrieve a response
   later. How is the response eventually retrieved?

There were mixed opinions about this in the meeting. While @pathakraul suggested to limit the API to messages without expecting a response, others suggested that the original intent was.

@atishp04 There were no mixed opinions in the PRS TG meeting. AFAIK, both myself and @pathakraul were suggesting the same thing that it is the responsibility of the message protocol to define appropriate mechanism to track posted messages.

The response retrieval mechanism is specific to the message protocol. Since MPXY is just an abstract interface, it just allows blocking/unblocking mechanisms to send a response. If the message protocol defines posted request, it must provide appropriate additional mechanism to retrieve that response.
For example: It can define another message type that can be send using send_message_with_response to retrieve the response to a previously sent msgID. This message type should have the ability to track previous msgID for which it is seeking response. A message protocol may also not support posted request at all. In that case, sbi_mpxy_send_message_without_response will be used only for messages that doesn't expect response for that specific message protocol. Currently, RPMI doesn't have any posted requests. So we don't have any examples to support the use case. The supervisor software can either poll or wait for SSE/MSI notification to retrieve the response.
As per our offline discussion, it may not be prudent to have a place holder API without a valid use case in hand. There are other three options suggested

1. Limit it to posted messages for which no response is expected
2. Correctly define response retrieval and not as a hack on top of send with response
3. Drop the send without response till a use case arises

The Option 1 is certainely better because the API is generic and abstract enough to allow message protocol specification to define its own mechanism such as notification or another message to track the status or response for a posted message.

How about a 4th Option where we only have single sbi_mpxy_send_message() call with an extra flags parameter where the bit[0] of flags parameter tells SBI implementation whether it is posted message or not ??

Sorry. I was away for the last week as I was travelling to India and got sick.
I did not see any further discussion on this. Any problems with this approach ?