Recommended practice for scvelo fit evaluation

[mxposed]

Hi everyone. I'm wondering what is the recommended route to evaluate if scvelo fit on a particular dataset is good, bad, can be improved or should be abandoned?

As I noticed in several questions here on github, the main thing is to examine the gene phase portraits: they are the source of any signal in RNA velocity estimations.
How many genes should I examine, how to pick them? Should I look at top via fit_likelihood? Or the top fits for my cell types, according to rank_velocity_genes? Or rank_dynamical_genes?

If I don't see any genes with signal and curvature, then it's clear the dataset won't give any good RNA velocity estimation.

What to do if I see some genes with curvature and some without? Should I increase or decrease number of HVG picked for analysis?
Should I remove some cell types?
Can I manually include or exclude genes for RNA velocity estimation?
When should I try the different modes of velocity (dynamical, stochastic, deterministic) estimation? In which situations it can help?

I would appreciate any advice,
Thank you

[WeilerP]

@mxposed, here is how I'd proceed (I'll try to address all your questions):

1. Look at the phase portraits of genes with the highest likelihood. These contribute the most to the overall fit. I am always checking that these genes exhibit the characteristic almond/football shape due to the temporal delay in the splicing dynamics. If you already have some prior knowledge on cell differentiation, you can also verify that the cell clusters are ordered correctly along the phase portrait.
2. If there are any phase portraits not exhibiting the almond/football shape and potentially violating the expected cell differentiation you have from prior knowledge, you can remove them and refit the model. If I have to do this too often/for too many genes, I do no longer use RNA velocity. You could try one of the kernels in CellRank if you are interested in fate decision, etc.
   If you do not pick up any relevant signal, you can also check if this is due to outlier clusters. These could, for example, be cell types disconnected from all others in the latent space embedding. You can check if the fit and phase portraits improve after excluding this cell type.

I do not believe/expect that adding HVG will improve your fit significantly (presuming you are considering a reasonable large number in the first place). I would expect you'd be adding more noise than valuable info if any.
As mentioned in the paper by Bergen et al., the deterministic and stochastic model lead to false estimates when the system under investigation has not reached its steady states. While these two modes are faster, you do not infer the full set of parameters and only rely on a subset of the data, namely the extreme quantiles in the phase portraits based on which the steady state ratio is inferred.