Skip to content

Commit

Permalink
Merge pull request #145 from dyzheng/v2.2.3
Browse files Browse the repository at this point in the history
V2.2.3
  • Loading branch information
dyzheng authored May 27, 2022
2 parents 21ac83f + 6364f7a commit c64bc25
Show file tree
Hide file tree
Showing 353 changed files with 25,507 additions and 6,354 deletions.
74 changes: 74 additions & 0 deletions .github/workflows/cuda.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: CUDA Test

on:
workflow_dispatch:

jobs:
start-runner:
name: Start self-hosted EC2 runner
runs-on: ubuntu-latest
outputs:
label: ${{ steps.start-ec2-runner.outputs.label }}
ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2
- name: Start EC2 runner
id: start-ec2-runner
uses: machulav/ec2-github-runner@v2
with:
mode: start
github-token: ${{ secrets.PAT }}
ec2-image-id: ami-04cd9fec4a7a39019
ec2-instance-type: g4dn.xlarge
subnet-id: subnet-72d3e53e
security-group-id: sg-06b0c93122c08aeab

test:
name: Do the job on the runner
needs: start-runner # required to start the main job when the runner is ready
runs-on: ${{ needs.start-runner.outputs.label }} # run the job on the newly created runner
container:
image: ghcr.io/deepmodeling/abacus-development-kit:cuda
options: --gpus all
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Build cuSolver
run: |
nvidia-smi
cmake -B build -DUSE_CUSOLVER_LCAO=ON
cmake --build build -j4
cmake --install build
- name: Test
run: |
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda/lib64
cd tests/integrate
echo "ks_solver cusolver" >> ./270_NO_MD_2O/INPUT
./Autotest.sh -r 270_NO_MD_2O
stop-runner:
name: Stop self-hosted EC2 runner
needs:
- start-runner # required to get output from the start-runner job
- test # required to wait when the main job is done
runs-on: ubuntu-latest
if: ${{ always() }} # required to stop the runner even if the error happened in the previous jobs
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2
- name: Stop EC2 runner
uses: machulav/ec2-github-runner@v2
with:
mode: stop
github-token: ${{ secrets.PAT }}
label: ${{ needs.start-runner.outputs.label }}
ec2-instance-id: ${{ needs.start-runner.outputs.ec2-instance-id }}
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:
jobs:
build_container_and_push:
runs-on: ubuntu-latest
if: github.repository_owner == 'deepmodeling'
strategy:
matrix:
dockerfile: ["gnu","intel","cuda"]
Expand All @@ -25,10 +26,19 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Login to Aliyun Registry
uses: docker/login-action@v1
with:
registry: ${{ secrets.DP_HARBOR_REGISTRY }}
username: ${{ secrets.DP_HARBOR_USERNAME }}
password: ${{ secrets.DP_HARBOR_PASSWORD }}

- name: Build and Push Container
uses: docker/build-push-action@v2
with:
tags: ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.dockerfile }}
tags: |
ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.dockerfile }}
${{ secrets.DP_HARBOR_REGISTRY }}/dplc/abacus-${{ matrix.dockerfile }}:latest
file: Dockerfile.${{ matrix.dockerfile }}
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{matrix.dockerfile}}
cache-to: type=inline
Expand Down
42 changes: 42 additions & 0 deletions .github/workflows/image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Build Image

on:
push:
branches:
- develop

jobs:
build_container_and_push:
runs-on: ubuntu-latest
if: github.repository_owner == 'deepmodeling'
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Login to Aliyun Registry
uses: docker/login-action@v1
with:
registry: ${{ secrets.DP_HARBOR_REGISTRY }}
username: ${{ secrets.DP_HARBOR_USERNAME }}
password: ${{ secrets.DP_HARBOR_PASSWORD }}

- name: Build and Push Container
uses: docker/build-push-action@v2
with:
tags: |
ghcr.io/${{ github.repository_owner }}/abacus:latest
${{ secrets.DP_HARBOR_REGISTRY }}/dplc/abacus:latest
file: Dockerfile
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/abacus:latest
cache-to: type=inline
push: true
34 changes: 34 additions & 0 deletions .github/workflows/performance.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Performance test

on:
workflow_dispatch:

jobs:
test:
name: Performance test
runs-on: self-hosted
strategy:
matrix:
tag: ["gnu", "intel"]
if: github.repository_owner == 'deepmodeling'
container: ghcr.io/${{ github.repository_owner }}/abacus-development-kit:${{ matrix.tag }}
timeout-minutes: 2880
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install Requirements
run: |
apt install -y time
- name: Build
run: |
cmake -B build -DENABLE_LIBXC=ON
cmake --build build -j16
cmake --install build
- name: Test
run: |
. /opt/intel/oneapi/setvars.sh 2> /dev/null || :
cd examples/performance/
bash run.sh
- name: Show Result
run: |
cat examples/performance/sumall.dat
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
ref: "refs/pull/${{ github.event.number }}/merge"
- name: Build
run: |
cmake -B build -DBUILD_TESTING=ON -DENABLE_DEEPKS=ON
cmake -B build -DBUILD_TESTING=ON -DENABLE_DEEPKS=ON -DENABLE_LIBXC=ON
cmake --build build -j8
cmake --install build
- name: Test
Expand Down
49 changes: 37 additions & 12 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
########################################
# CMake build system
# This file is part of ABACUS
cmake_minimum_required(VERSION 3.18)
cmake_minimum_required(VERSION 3.16)
########################################

project(ABACUS
Expand All @@ -13,15 +13,16 @@ project(ABACUS

option(ENABLE_DEEPKS "Enable DeePKS functionality" OFF)
option(ENABLE_LIBXC "Enable LibXC functionality" OFF)
option(USE_CUDA "Enable support to CUDA." OFF)
option(USE_CUDA "Enable support to CUDA for PW." OFF)
option(USE_CUSOLVER_LCAO "Enable support to CUSOLVER for LCAO." OFF)
option(USE_ROCM "Enable support to ROCm." OFF)
option(USE_OPENMP " Enable OpenMP in abacus." ON)
option(ENABLE_ASAN "Enable AddressSanitizer" OFF)
option(BUILD_TESTING "Build ABACUS unit tests" OFF)
option(GENERATE_TEST_REPORTS "Enable test report generation" OFF)

set(ABACUS_BIN_NAME abacus)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/modules)
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/modules)
set(ABACUS_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/source)
set(ABACUS_TEST_DIR ${CMAKE_CURRENT_SOURCE_DIR}/tests)
set(ABACUS_BIN_PATH ${CMAKE_CURRENT_BINARY_DIR}/${ABACUS_BIN_NAME})
Expand Down Expand Up @@ -58,21 +59,33 @@ set(CMAKE_CXX_STANDARD 11)
include(CheckLanguage)
check_language(CUDA)
if(CMAKE_CUDA_COMPILER)
if(NOT DEFINED USE_CUDA)
message("CUDA components detected. \nWill build the CUDA version of ABACUS.")
set(USE_CUDA ON)
if(NOT DEFINED USE_CUDA OR NOT DEFINED USE_CUSOLVER_LCAO)
if (NOT DEFINED USE_CUDA AND NOT DEFINED USE_CUSOLVER_LCAO)
message("CUDA components detected. \nWill build the CUDA for PW version of ABACUS by default.")
set(USE_CUDA ON)
set(USE_CUSOLVER_LCAO OFF)
elseif (NOT DEFINED USE_CUDA)
set(USE_CUDA OFF)
else()
set(USE_CUSOLVER_LCAO OFF)
endif()
else()
if(NOT USE_CUDA)
message(WARNING "CUDA components detected, but USE_CUDA set to OFF. \nNOT building CUDA version of ABACUS.")
if(NOT USE_CUDA AND NOT USE_CUSOLVER_LCAO)
message(WARNING "CUDA components detected, but both USE_CUDA and USE_CUSOLVER_LCAO set to OFF. \nNOT building CUDA version of ABACUS.")
elseif (USE_CUDA AND USE_CUSOLVER_LCAO)
message(FATAL_ERROR "USE_CUDA and USE_CUSOLVER_LCAO set, but now they not allowed to coexist.")
endif()
endif()
else() # CUDA not found
if (USE_CUDA)
message(FATAL_ERROR "USE_CUDA set but no CUDA components found.")
if (USE_CUDA OR USE_CUSOLVER_LCAO)
message(FATAL_ERROR "USE_CUDA or USE_CUSOLVER_LCAO set but no CUDA components found.")
set(USE_CUDA OFF)
set(USE_CUSOLVER_LCAO OFF)
endif()
endif()
if(USE_CUDA)

if(USE_CUDA OR USE_CUSOLVER_LCAO)
cmake_minimum_required(VERSION 3.18) # required by `CUDA_ARCHITECTURES` below
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_EXTENSIONS ON)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
Expand All @@ -92,11 +105,18 @@ if(USE_CUDA)
60 # P100
70 # V100
75 # T4
80 # A100
)
include_directories(${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
add_compile_definitions(__CUDA)
if (USE_CUDA)
add_compile_definitions(__CUDA)
endif()
if (USE_CUSOLVER_LCAO)
add_compile_definitions(__CUSOLVER_LCAO)
endif()
endif()


# Warning: CMake add support to HIP in version 3.21. This is rather a new version.
# Use cmake with AMD-ROCm: https://rocmdocs.amd.com/en/latest/Installation_Guide/Using-CMake-with-AMD-ROCm.html
if(USE_ROCM)
Expand Down Expand Up @@ -263,6 +283,7 @@ target_link_libraries(${ABACUS_BIN_NAME}
cell
symmetry
md
planewave
surchem
neighbor
orb
Expand All @@ -276,6 +297,10 @@ target_link_libraries(${ABACUS_BIN_NAME}
ri
driver
xc
hsolver
elecstate
hamilt
psi
esolver
-lm
)
Expand Down
17 changes: 17 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# To build this Dockerfile, run `docker build -t abacus - < Dockerfile`.
# Pull image with `docker pull ghcr.io/deepmodeling/abacus:latest`.
FROM ubuntu:latest
RUN apt update && apt install -y --no-install-recommends libopenblas-dev liblapack-dev libscalapack-mpi-dev libelpa-dev libfftw3-dev libcereal-dev libxc-dev g++ make cmake bc time sudo vim git
# If you wish to use the LLVM compiler, replace 'g++' above with 'clang libomp-dev'.
RUN GIT_SSL_NO_VERIFY=true git clone https://github.com/deepmodeling/abacus-develop.git --depth 1 && cd abacus-develop && cmake -B build && cmake --build build -j`nproc` && cmake --install build && cd .. && rm -rf abacus-develop
# If you have trouble cloning repo, replace "github.com" with "gitee.com".
ENV OMPI_ALLOW_RUN_AS_ROOT=1 OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 OMPI_MCA_btl_vader_single_copy_mechanism=none
CMD mpirun --use-hwthread-cpus abacus

# To run ABACUS built by this image with all available threads, execute `docker run -v <host>:<wd> -w <wd/input> abacus`.
# Replace '<host>' with the path to all files(including pseudopotential files), '<wd>' with a path to working directory, and '<wd/input>' with the path to input folder(containing 'INPUT', 'STRU', etc.).
# e.g. after clone the repo to `$HOME` and pulled this image, execute `docker run -v ~/abacus-develop/tests/integrate:/workspace -w /workspace/101_PW_15_f_pseudopots abacus`.
# To run ABACUS with a given MPI process number, execute `docker run -v <host>:<wd> -w <wd/input> -it --entrypoint mpirun abacus -np <processes> abacus`. Note: the first "abacus" is the name of the image, the second "abacus" is the name of the executable file. Do not use '--cpus' flag of 'docker run' to specify the number of processes.

# To use this image as developing environment, execute `docker run -it --entrypoint /bin/bash abacus`.
# Please refer to https://docs.docker.com/engine/reference/commandline/run/ for more details.
6 changes: 3 additions & 3 deletions Dockerfile.cuda
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM ghcr.io/deepmodeling/abacus-development-kit:gnu

RUN wget https://developer.download.nvidia.com/compute/cuda/11.4.2/local_installers/cuda_11.4.2_470.57.02_linux.run --no-check-certificate --quiet \
&& sh cuda_11.4.2_470.57.02_linux.run --toolkit --silent \
&& rm cuda_11.4.2_470.57.02_linux.run
RUN wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run --no-check-certificate --quiet \
&& sh cuda_11.7.0_515.43.04_linux.run --toolkit --silent \
&& rm cuda_11.7.0_515.43.04_linux.run
ENV PATH=/usr/local/cuda/bin:$PATH
2 changes: 1 addition & 1 deletion docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ This file provides a guideline for it.
## Table of Contents

- [Got a question?](#got-a-question)
- [Before getting started](#before-getting-started)
- [Structure of the package](#structure-of-the-package)
- [Submitting an Issue](#submitting-an-issue)
- [Comment Style for documentation](#comment-style-for-documentation)
Expand All @@ -23,6 +22,7 @@ If you would like to implement a new feature, please submit an issue with a prop

## Structure of the package

Please refer to [our instructions](./install.md) on how to installing ABACUS.
The source code of ABACUS is based on several modules. Under the ABACUS root directory, there are the following folders:

- `cmake`: relevant files for finding required packages when compiling the code with cmake;
Expand Down
2 changes: 1 addition & 1 deletion docs/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ equation. For PW basis, there are CG and Blocked Davidson methods for solving th
equation for each basis.

- PW: ks_solver = ‘cg’ or ‘dav’
- LCAO: ks_solver = ‘hpseps’ , ‘genelpa’ or ‘lapack’
- LCAO: ks_solver = ‘hpseps’ , ‘genelpa’ , ‘scalapack_gvx’ or 'cusolver'
- LCAO_in_PW: ks_solver = ‘lapack’

If you set ks_solver=‘hpseps’ for basis_type=‘pw’, the program will be stopped with an error
Expand Down
Loading

0 comments on commit c64bc25

Please sign in to comment.