[Bug] triaxial Driver cannot be running with develop repo

[jhuang2601]

Describe the bug
Reran a triaxialDriver example with latest develop repo and it does not work any more. The case was up and running previously and something must be broken.

To Reproduce
Steps to reproduce the behavior:
The xml file for the testing example is attached below:
case.zip
Run it with the develop repo to reproduce the reported issue

Screenshots

Max threads: 2
MKL max threads: 1
GEOSX version 0.2.0 (HEAD, sha1: bb16d72)
Adding Event: SoloEvent, triaxialDriver
   TableFunction: strainFunction
   TableFunction: stressFunction
Adding Mesh: InternalMesh, mesh1
Total number of nodes:8
Total number of elems:1
Rank 0: Total number of nodes:8
***** ERROR
***** LOCATION: /tmp/GEOSX/src/coreComponents/mesh/FaceManager.cpp:199
***** Controlling expression (should be false): true
***** Rank 0: Face 3 is not connected to an element.

** StackTrace of 7 frames **
Frame 0: /data/gpfs/Users/j0436735/travis-deployments/CPU/GEOSX-bb16d72/bin/geosx 
Frame 1: /apps/intel/2019/u5/compilers_and_libraries_2019.5.281/linux/compiler/lib/intel64_lin/libiomp5.so 
Frame 2: __kmp_invoke_microtask 
Frame 3: /apps/intel/2019/u5/compilers_and_libraries_2019.5.281/linux/compiler/lib/intel64_lin/libiomp5.so 
Frame 4: /apps/intel/2019/u5/compilers_and_libraries_2019.5.281/linux/compiler/lib/intel64_lin/libiomp5.so 
Frame 5: /apps/intel/2019/u5/compilers_and_libraries_2019.5.281/linux/compiler/lib/intel64_lin/libiomp5.so 
Frame 6: /lib64/libpthread.so.0 
Frame 7: clone 
=====

***** ERROR
***** LOCATION: /tmp/GEOSX/src/coreComponents/mesh/FaceManager.cpp:199
***** Controlling expression (should be false): true
***** Rank 0: Face 0 is not connected to an element.

** StackTrace of 9 frames **
Frame 0: /data/gpfs/Users/j0436735/travis-deployments/CPU/GEOSX-bb16d72/bin/geosx 
Frame 1: GOMP_parallel 
Frame 2: geosx::FaceManager::sortAllFaceNodes(geosx::NodeManager const&, geosx::ElementRegionManager const&) 
Frame 3: geosx::DomainPartition::setupCommunications(bool) 
Frame 4: geosx::ProblemManager::generateMesh() 
Frame 5: geosx::ProblemManager::problemSetup() 
Frame 6: geosx::GeosxState::initializeDataRepository() 
Frame 7: main 
Frame 8: __libc_start_main 
Frame 9: /data/gpfs/Users/j0436735/travis-deployments/CPU/GEOSX-bb16d72/bin/geosx 
=====

Platform (please complete the following information):

• Machine: Pecan
• GEOSX Version: develop bb16d72

[klevzoff]

Due to some of the changes in #1847, it is now necessary to define at least one (dummy) CellElementRegion along with the mesh, such that each face is adjacent to an element in some region.

Previously, the mesh generation process would partially populate the face-to-element maps, such that even in absence of regions, in would contain some element indices, and this check would pass:
https://github.com/GEOSX/GEOSX/blob/bb16d72e8dc43a94f3f55e9b6aa6dc7de003f669/src/coreComponents/mesh/FaceManager.cpp#L197-L200

With the new algorithm, the maps remain completely empty (i.e. -1s) if no regions are present, and the check fails. In my opinion, we should not require/assume that each face is connected to a valid element. For example, in meshes like "staircase" we currently define both "Channel" and "Barrier" regions so that all original Cartesian faces are included, but in principle we should drop "Barrier" as it does not contribute to the solution.

If everyone is on board with the changes, removing the above check should be an easy fix. Care needs to be taken to ensure other components of the code (e.g. stencils) are also capable of handling (skipping) unconnected faces. Finally, it would also be helpful to try eliminate the need for a dummy mesh in TriaxialDriver inputs files.

[TotoGaz]

If everyone is on board with the changes, removing the above check should be an easy fix.

But are those checks legit when it comes to real simulation situations? I think they do?

but in principle we should drop "Barrier" as it does not contribute to the solution.

In principle we could also argue that

• we should not give GEOSX elements it will not use.
• we could enhance InternalMeshGenerator to get rid of the unused elements?

I agree that this can be convenient anyway.

Also, could we add the missing CellElementRegion to the test (looks like that's what is done for some of those tests)?

[klevzoff]

In principle we could also argue that

• we should not give GEOSX elements it will not use.
• we could enhance InternalMeshGenerator to get rid of the unused elements?

The problem (if I can call it that) lies in the separation of responsibilities: mesh generators produce all cell blocks that were requested, along with other adjacent entities (faces, nodes, etc.), and the decision on what is used and what isn't is made later by the user defining element regions. We could of course collect a list of "used" cell block names beforehand, and pass them to generateMesh() call. It has the potential of complicating mesh generators a bit, for example mesh nodes that remain unused need to be deleted and remaining ones compressed and renumbered in order to preserve continuous local indexing. But it's a nice optimization opportunity in terms of time and memory (and even the size of output files).

[klevzoff]

But are those checks legit when it comes to real simulation situations? I think they do?

We can safely skip faces/edges/nodes that are not attached to elements regions. Most code paths will already handle this correctly (e.g. DofManager does not assign dof indices to these). In case of the check above, it doesn't need to rearrange nodes of unused faces since those will not be used anywhere, so it could just continue instead of error.

[jhuang2601]

By adding the following ElementRegions block, now the examples can be running:

  <ElementRegions>
    <CellElementRegion
      name="dummy"
      cellBlocks="{ cellBlock01 }"
      materialList="{ dummy }"/>
  </ElementRegions>

[TotoGaz]

Great that it works @jhuang2601

@klevzoff Could the "real problem" also be that our "dummy" mesh is not that "dummy"? It's one hex cell.
Maybe providing a real empty mesh, with 0 nodes, cells, faces, cell blocks... could be interesting. All the loops/code downstream this mesh should be OK (you start looping on the faces, but there is none, you do nothing, that's fine)

[jhuang2601]

Solution is found for running this case. Long term solution for how to remove "dummy" mesh has not been decided.