BUGS when installing UMEP on QGIS 3.16.8

[ML426]

Python error : Unable to load the 'processing_umep' extension causes an error when calling its classFactory() method See the Python error log for more details

2021-08-09T10:16:41 WARNING warning:C:\PROGRA1/QGIS311.8/apps/qgis-ltr/./python\qgis\utils.py:793: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
mod = _builtin_import(name, globals, locals, fromlist, level)

         traceback: File "", line 1, in 
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 354, in startPlugin
          if not _startPlugin(packageName):
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 334, in _startPlugin
          plugins[packageName] = package.classFactory(iface)
          File "C:\Users/XXXX/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\__init__.py", line 34, in classFactory
          from .UMEP import UMEP
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
          mod = _builtin_import(name, globals, locals, fromlist, level)
          File "", line 1007, in _find_and_load
          File "", line 986, in _find_and_load_unlocked
          File "", line 680, in _load_unlocked
          File "", line 855, in exec_module
          File "", line 228, in _call_with_frames_removed
          File "C:\Users/XXXX/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\UMEP.py", line 32, in 
          from .SkyViewFactorCalculator.svf_calculator import SkyViewFactorCalculator
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
          mod = _builtin_import(name, globals, locals, fromlist, level)
          File "", line 1007, in _find_and_load
          File "", line 986, in _find_and_load_unlocked
          File "", line 680, in _load_unlocked
          File "", line 855, in exec_module
          File "", line 228, in _call_with_frames_removed
          File "C:\Users/XXXX/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\SkyViewFactorCalculator\svf_calculator.py", line 24, in 
          from future import standard_library
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
          mod = _builtin_import(name, globals, locals, fromlist, level)
          File "", line 1058, in _handle_fromlist
          File "", line 228, in _call_with_frames_removed
          File "", line 1007, in _find_and_load
          File "", line 986, in _find_and_load_unlocked
          File "", line 680, in _load_unlocked
          File "", line 855, in exec_module
          File "", line 228, in _call_with_frames_removed
          File "C:\PROGRA~1\QGIS31~1.8\apps\Python39\lib\site-packages\future\standard_library\__init__.py", line 65, in 
          import imp
          File "C:\PROGRA~1/QGIS31~1.8/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
          mod = _builtin_import(name, globals, locals, fromlist, level)
          File "", line 1007, in _find_and_load
          File "", line 986, in _find_and_load_unlocked
          File "", line 680, in _load_unlocked
          File "", line 855, in exec_module
          File "", line 228, in _call_with_frames_removed
          File "C:\PROGRA~1\QGIS31~1.8\apps\Python39\lib\imp.py", line 31, in 
          warnings.warn("the imp module is deprecated in favour of importlib; "

[biglimp]

I see you also submitted an issue. Lets start here and then move over to the issue if needed. I need more info. What operating system and what version of UMEP are you trying to install. I presume it is processing for UMEP?

[ML426]

When I restart Qgis, and chosse "NOT intall the UMEP", I still get the same warnings (on the log tab) and I still have the UMEP tab on Qgis.
On the other hand, when I do a test with the pre-processor, to import the meteorological data (ERA 5), I get an error message : This plugin requires the supy package to be installed OR upgraded. Please consult the FAQ in the manual for further information on how to install missing python packages

[biglimp]

If you try e.g. the Shadow Generator tool, do you still get an error message?

[ML426]

Yes, i have a error message

[biglimp]

Same error message?

[ML426]

2021-08-11T16:14:32 WARNING Traceback (most recent call last):
File "C:\Users/mmuandal/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\DSMGenerator\dsm_generator.py", line 312, in start_progress
dem_layer_crs.ImportFromWkt(demRaster.GetProjection())
AttributeError: 'NoneType' object has no attribute 'GetProjection'

[ML426]

not the same message error

[sunt05]

Hi @ML426, please do python -m pip install supy --user in your conda environment and see if things are working.

[ML426]

Hi @sunt05,
Thanks, the first step is ok :
Successfully installed algopy-0.5.7 argparse-1.4.0 asteval-0.9.25 atmosp-0.2.9 cdsapi-0.5.1 dask-2021.7.2 dill-0.3.4 f90nml-1.3.1 fsspec-2021.7.0 linecache2-1.0.0 lmfit-1.0.2 locket-0.2.1 multiprocess-0.70.12.2 nose-1.3.7 numdifftools-0.9.40 partd-1.2.0 patsy-0.5.1 pint-0.17 platypus-opt-1.0.4 pvlib-0.8.1 seaborn-0.11.1 statsmodels 0.12.2 supy-2021.7.22 supy-driver-2021a2 toolz-0.11.1 tqdm-4.62.0 traceback2-1.4.0 uncertainties-3.1.6 unittest2 1.1.0 xarray-0.19.0

But now a have a other error message :
This plugin requires that you have configured your computer to use the CDS API.

[sunt05]

Great!
You are very close to a successful run.
We might have missed to include the following instruction in our plugin; but please follow the link below to set up your CDS API:

https://cds.climate.copernicus.eu/api-how-to

[ML426]

Thank you for your help.
After 3 days, I finally found the solution.
I am grateful to @biglimp and @sunt05 .

[ML426]

I hope i'll not have the same problems for SkyViewFactor

[rarygit]

I think, it didn't found some packages.
I'm using Qgis from anaconda platform (but i had the same problem without anaconda)

Although this issue has been answered in the Discussion, I have some comments regarding using a conda environment in both Windows and Linux for QGIS and UMEP.

I always use miniconda (minimal conda), and never Anaconda as the package manager.
Although Anaconda has a nice shiny interface, I often found it throwing conflicts when installing packages
From my experience in minimising conflicts, I recommend conda-forge as the main channel.

Regarding pip, only use pip to install a package if there is no other option and only after you have set up your environment exactly as you need it. You want to avoid breaking your environment and ending up with package conflicts.

Creating a new miniconda environment is the best way to experiment and learn how to use conda.
If you make a mistake, you can remove the environment.

For newcomers to miniconda, these webpages are indispensable:
https://docs.conda.io/en/latest/miniconda.html
https://docs.conda.io/projects/conda/en/latest/user-guide/install/download.html#anaconda-or-miniconda
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
https://docs.conda.io/projects/conda/en/latest/_downloads/843d9e0198f2a193a3484886fa28163c/conda-cheatsheet.pdf

I run a separate miniconda environment each for QGIS v3.16 LTR, v3.18 and next will be another for v3.20
I use the QGIS v3.18 to run the plugin gis4wrf and avoid netcdf conflicts.
The QGIS v3.16 LTR environment is for production and stability.
The QGIS v3.20 environment will be to experiment with new features.

It is easy and straight forward to run your all your conda environments from a separate fast SSD disk (e.g. SanDisk or other) that is not your Windows or Linux OS drive. There are tutorials on the internet for this. I found it easier under Ubuntu linux.

In other miniconda environments I am able to compile from source, run cuda code, implement other environments from a .yml text file .... etc

If one QGIS environment breaks or fails for some reason, then remove it and setup a new minimal conda environment.

That being said, there is a world of difference running miniconda on Ubuntu compared to the Windows file management system.

[rarygit]

Just for completion, I have attached two yml files that illustrate the two miniconda environments I created for QGIS v3.16 and v3.18
That is, I do not install QGIS as an Ubuntu standalone software. Or by using OSGEO on Windows.
Instead I first install a minimal conda (miniconda).
https://docs.conda.io/en/latest/miniconda.html#linux-installers
Then I install QGIS version (your choice) inside miniconda as an environment (e.g. qgis3_16)
Each environment is an autonomous python environment - if it breaks you can delete it and simply:
conda create -n qgis3_16
conda activate qgis3_16

Now stay in this activated environment to install your version of qgis

Then read here to understand installing qgis from the conda-forge channel:
https://github.com/conda-forge/qgis-feedstock
"Installing qgis from the conda-forge channel can be achieved by adding conda-forge to your channels with:"
conda config --add channels conda-forge
conda config --set channel_priority strict
Once the conda-forge channel has been enabled, qgis can be installed with:
conda install qgis
or
conda install qgis=3.16

Generally I then install the tricky python packages immediately after I have installed qgis
Such as matplotlib, netcdf ... and the others which you may encounter with Anaconda
Once everything is finalised and tested OK in your qgis environment (i.e. you actually started qgis from the terminal, inside your conda qgis environment), install pip
conda install pip
Then install supy using pip, not conda, inside your conda qgis environment
(although having a conda-forge supy-feedstock available to install supy would be a great feature ....)
Then reopen QGIS inside the already activated qgis conda environment:
qgis
Once opened, install UMEP plugin using the QGIS plugin manager.

The point is to experiment with different environments until you feel that the python packages are optimal for your purpose or research.
Then activate the conda environment
conda activate qgis3_16
and to stop it:
conda deactivate

For more details, you should read this conda user manual page:
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#cloning-an-environment

Comments:
OS is Ubuntu 20.04
For QGIS, I don't install via Anaconda; only use miniconda on Linux and Windows
Ubuntu OS (partition#1 for OS) and both QGIS conda environments (partition#2 for conda env's and compiling) run from a 240Gb SanDisk SSD with two partitions. The performance improvement is noticeable.
QGIS v3.16: Ignore the warning that the plugin giswrf isn't loaded.
I am using a different netcdf package and I don't want to break UMEP
QGIS 3.18 loads gis4wrf, this is where I want to use it
You can try using using these yml files to set up the same environment on Ubuntu 20.04
https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment

Hope this example helps newcomers to QGIS on miniconda.

qgis_conda_envs.zip

[ML426]

I had done the same thing. With anaconda, I created a new environment specially for Qgis, I installed the Qgis version I was interested in (Qgis 3.16) then I installed the missing plugins.
This link can also help : https://anaconda.org/conda-forge/qgis

Thanks to the team for your help
:)

[aguosha]

Hi. I'm a beginner level QGIS user. I don't use it through anaconda (sys.executable: 'C:\Program Files\QGIS 3.16\bin\qgis-ltr-bin.exe').

When I tried to install UMEP plug-in, I got the following message:

Couldn't load plugin 'UMEP' due to an error when calling its classFactory() method 

ModuleNotFoundError: No module named 'packaging' 
Traceback (most recent call last):
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 334, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\__init__.py", line 34, in classFactory
    from .UMEP import UMEP
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\UMEP.py", line 50, in 
    from .SuewsSimple.suews_simple import SuewsSimple
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\SuewsSimple\suews_simple.py", line 35, in 
    from ..suewsmodel import suews_wrapper
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\suewsmodel\__init__.py", line 5, in 
    from .supy_installer import setup_supy
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\suewsmodel\supy_installer.py", line 4, in 
    from packaging import version
  File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'packaging'


Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] 
QGIS version: 3.16.7-Hannover Hannover, 0471743889 

Python Path:
C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\LAStools
C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python
C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python
C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins
C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python/plugins
C:\PROGRA~1\QGIS3~1.16\apps\Python37
C:\PROGRA~1\QGIS3~1.16\apps\Python37\Scripts
C:\Program Files\QGIS 3.16\bin\python37.zip
C:\PROGRA~1\QGIS3~1.16\apps\Python37\DLLs
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib
C:\Program Files\QGIS 3.16\bin
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\win32
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\win32\lib
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\Pythonwin
C:/Users/Aga/AppData/Roaming/QGIS/QGIS3\profiles\default/python

[rarygit]

It seems that you need to install the python module called "packaging".

Although not related to QGIS/UMEP, there is a discussion on Stack Exchange regarding this module:
https://stackoverflow.com/questions/42222096/no-module-named-packaging

[biglimp]

We are in transition between QGIS 3.16 and QGIS 3.20 which make use of different Python versions (3.7 and 3.9). That might be why you have issues. I see that packaging now is available from the installation repository:

Visit https://umep-docs.readthedocs.io/en/latest/Getting_Started.html#adding-missing-python-libraries-and-other-osgeo-functionalities and see if you can install packaging on your system.