Skip to content

Latest commit

 

History

History
597 lines (504 loc) · 26 KB

README.md

File metadata and controls

597 lines (504 loc) · 26 KB
Raw

SCRAM

A Software Configuration, Release and Management tool which supports configuration management and management processes for software development.

Synopsis

scram [options] <COMMANDS> [sub-commands] [command-options]

Description

SCRAM is a software engineering tool, that supports the configuration management and management processes for software development. It resolves the issues of configuration definition, build, project organization and run-time environment. It was designed with a focus on supporting a distributed, multi-project development work-model

Options

  • -a, --arch <arch>
    Override the SCRAM_ARCH environment variable value.
  • -d, --debug Turns on any debug output.
  • -h, --help Shows the scram help message. You can also run "scram build help" in your development area to get project specific build targets help message.

Scram Commands

  • Common Options
    All scram commands recognize following options.

  • -h, --help To show the help message for the selected command e.g.

    scram list -h

COMMANDS

arch

Show the current architecture flag. If this command is run from a SCRAM-based release/developer area then it shows the architecture for which that release was created otherwise shows out the SCRAM_ARCH environment variable value.

build [options] [gmake-options] [<targets>]

Build the productions (shared libraries, plugins, executables and tests) for the current project area.

OPTIONS

  • -c, --convertxml Obsolete: Convert any old-style BuildFile in to new BuildFile.xml
  • -f, --fast Skip checking the internal SCRAM cache update and go straight to build/gamke
  • --ignore-arch Avoid SCRAM warning about architecture mismatch e.g. compiling on SLC5 machine an slc6_* based release.
  • -r, --reset Reset/re-generate all internal SCRAM caches for your development area.
  • -t, --testrun Do not run gmake but only do any internal SCRAM caches updates.
  • -v, --verbose For more verbose output from the underlaying gmake process

GMAKE OPTIONS

You can pass any gmake command-line option which does not conflicit with 'scram build' options. e.g.

  • -j [N], --jobs[=N] Allow N jobs at once; infinite jobs with missing N
  • -k, --keep-going Keep going when some targets can't be made

BUILD TARGETS

  • help
    Shows what SCRAM will build for your current working directory. You can run "scram build help" in your project's top-level directory to get help on extra build targets your project supports.

  • clean
    Clean up all the products built/generated by SCRAM.

  • python
    To just build python i.e. creating sym-links and compiling python modules.

  • runtests
    Build and run all tests for the project, or package-level tests only. For each test SCRAM create an individual log file tmp/SCRAM_ARCH/src/Subsystem/Package/test/<TestName>/testing.log. In order to run a single test you can run 'scram b runtests_<TestName>'.

  • unittests
    Same as runtests but does not try to build the test first.

  • checker
    Run LLVM static checkers

  • dxr
    Generate DXR indexes for the release.

  • gindices
    Generate glimpse indexes for the release.

  • enable-biglib disable-biglib check-biglib
    Disable/enable/check big EDM plugins

  • enable-multi-targets disable-multi-targets check-multi-targets
    Disable/enable/check multi-archs build support.

  • updateclassversion
    Run the edmCheckClassVersion and generate new src/<package>/classes_def.xml.generated file.

  • prebuild postbuild
    Execute any pre/post-build target for the project e.g. generating various source file for root/lcg dictionaries etc.

  • src release release-build all project
    Build all products (shared libs, plugins, binaries) for the project

  • <path>
    Build all products (shared libs, plugins, binaries) and their dependencies for src/<path>

  • runtime
    Dump the runtime environment used at build time.

  • ProjectRename projectrename PROJECTRENAME
    Fix the project area which was move/renamed after its creation.

  • ExternalLinks ToolUpdated
    Re-create symblinks for externals tools.

  • echo_<tool|package>_USED_BY
    Shows what depends on <tool> or <package> e.g.

    scram build echo_boost_USED_BY scram build echo_FWCore/Framework_USED_BY

    should show what depend on boost and FWCore/Framework

  • echo_<tool|package>_USES
    Shows what is the dependency of <tool> or <package>

  • echo_<productname>_ORIGIN
    Shows from which package <productname> was built.

  • echo_<variable>
    Shows the value of a variable used by the game e.g.

    scram build echo_CXXFLAGS

    shows the default flags used by the CXX compilers

config [parameter[=value]]

Show/Set site specific parameters.Running it without any argument shows all the available parameters and their values for your site.

OPTIONS

  • paramter
    Shows current and valid values for paramter.

  • paramter=value
    Set new value for the paramter.

    Supported site configuration parameters are

    • release-checks=1|0|yes|no
      Enable/disable release checks e.g. production architectures, deprecated releases. This avoids accessing releases information from internet. Default value is 1.
    • release-checks-timeout=[3-9]|[1-9][0-9]+
      Time in seconds after which a request to get release information should be timed out (min. value 3s). Default value is 10s.
    • scram-target=none|default|auto
      For CMSSW_14 and above, this setting can update the SCRAM_TARGET at install time. This is helpfull if you want to change SCRAM_TARGET=default tp0 SCRAM_TARGET=auto to enable the automatic selection of best micro architecture. By default CMSSW_14 and above are built with SCRAM_TARGET=default.
    • scram-post-install-script=/path/to/script
      For CMSSW_14 and above, for all scram based projects "scram-post-install-script" will run to do any site specific updates to project.
    • cmssw-post-install-script=/path/to/script
      For CMSSW_14 and above, for all cmssw type projects e.g. cmssw and cmssw-patch "cmssw-post-install-script" will run to do any site specific updates to project.

db --show|--link <path>|--unlink <path>

Show, link/add or unlink/remove additional SCRAM db. By adding an external SCRAM db in your local scram db allows scram to let you create developers area for projects/releases available in other cms installation. Your local scram db is available under $CMS_PATH/etc/scramrc/links.db

OPTIONS

  • -l , --link <path>
    Link/Add an external scram db <path> in to local scram db.
  • -s, --show
    Show all the external databases linked in to your SCRAM db.
  • -u, --unlink <path>
    Unlink/Remove an already linked external db path from the local scram db.

list [options] [<project_name>] [<version>]

Show available SCRAM-based projects for the selected SCRAM_ARCH.

OPTIONS

  • -A, --all Show projects for all available architectures.
  • -c --compact
    Show project list in compact format. Output format of each line will be ProjectName Version ReleaseInstallPath
  • -e, --exists
    Show only projects will actually looks OK. Note, this might be slow on distributed filesystems as SCRAM has to check each installed project and its version.
  • project_name Optional: Name of the project for which SCRAM should display the available versions.
  • version Optional: To Show only those installed versions which contain version

project [options] <-boot bootstrap_file> | <project_name version> | <release_path>

Creates a project developer area based on a release area or a release area using the project definition from the <bootstrap_file>. You can find the available releases by running 'scram list' command.

OPTIONS

  • -d, --dir <path>
    Indicate a project installation area into which the new project area should appear. Default is the current working directory.

  • -f, --force
    Force creation of developer area without checking for production architecture and deprecated release information. This avoid accessing releases information via internet.

  • -l, --log
    See the detail log message while creating a dev area.

  • -n, --name <name>
    Specify the name of the SCRAM-base development area you wish to create. By default version is used.

  • -s, --symlinks
    Creates symlinks for various product build directories e.g. lib/bin/tmp. You need to have ~/.scramrc/symlinks file to configure the symlink creation e.g. something like the following in the ~/.scramrc/symlinks file

    lib:/tmp/$(USER)/path
    tmp:/tmp/$(USER)/path
    will create

    /tmp/$(USER)/path/lib.<dummyname> -> $(LOCALTOP)/lib
    /tmp/$(USER)/path/tmp.<dummyname> -> $(LOCALTOP/tmp
    You can use $(SCRAM_PROJECTNAME) and $(SCRAM_PROJECTVERSION) in the .scramrc/symlinks file to create separate tmp areas for different projects.

  • -b, --boot <bootstrap_file>
    Creates a release area using the bootstrap file <bootstrap_file>.

  • [<project_name>] version
    Creates a developer area based on an already available release <version>.

  • <release_path>
    Creates a developer area based on <release_path> release area.

runtime -csh|-sh|-win [--dump <file>]

Shows the list of shell commands needed to set the runtime environment for the release. This command needs to be run from a release or developer area.

For SCRAM V3 based projects only, if there is project environment (e.g. LANG, LC_CTYPE ) which should not override the user env then please update ~/.scramrc/runtime file and add entries like

ignore: LANG
ignore: LC_CTYPE LC_ALL

You can eval the output of this command to set the runtime environment e.g.
eval `scram runtime -sh`

OPTIONS

  • -csh
    Show runtime commands for csh/tcsh shell.
  • -sh
    Show runtime commands for sh/bash/zsh shell.
  • -win
    Show runtime commands for cygwin.
  • --dump <file>
    Optional: Save the runtime environment in a <file> for the selected shell.

setup [<toolname>|<toolfile>.xml]

Setup/add an external tool to be used by the project. All selected tools definitions exists in config/toolbox/$SCRAM_ARCH/tools/selected directory in your project area. In order to change the tool definition, modify the corresponding xml toolfile and run scram setup <tool> command.

This command needs to be run from a release or developer area.

OPTIONS

  • <toolname>
    Name of the external tool which is already by the scram. A corresponding toolname.xml should exists under config/toolbox/$SCRAM_ARCH/tools.
  • <toolfile.xml> Full path of the toolfile. SCRAM will make a copy of toolfile.xml in to config/toolbox/$SCRAM_ARCH/tools for future use.

tool

This command consists of many sub-commands which deals with externals tools. This command needs to be run from a release or developer area.

tool list

Shows the list of all selected tools names and their versions.

tool info <toolname>

Shows the current tool definition for tool <toolname>.

tool remove <toolname>

Makes the tool <toolname> unavailable. SCRAM moves the tools definition file from config/toolbox/$SCRAM_ARCH/tools/selected/<toolname>.xml to config/toolbox/$SCRAM_ARCH/tools/available directory. So if one needs to select this tool again then just run 'scram setup <toolname>' command.

tool tag <toolname> <tag_name>

Shows the value of a variable(tag_name) for the toolname e.g.

      scram tool tag gcc-cxxcompiler CXX

  shows the c++ compiler's path.

unsetenv -csh|-sh|-win

Shows the list of shell commands needed to unset the runtime environment set previously by 'scram runtime command'. You can eval the output of this command to cleanup your previously set scram environment e.g. eval `scram unsetenv -sh`

OPTIONS

  • -csh
    Show unset commands for csh/tcsh shell.
  • -sh
    Show unset commands for sh/bash/zsh shell.
  • -win
    Show unset commands for cygwin.

version

Show default version of SCRAM. If this command is run from a SCRAM-based release/developer area then it shows the latest available version of SCRAM available for that release.

Scram Flags

SCRAM-based projects' build rules provided via cmssw-config support many compilation/control flags. Scope of build/compilation flag are defined by the place they are defined in. e.g.

  • Project level flags
    Any flag defined/provided via following are used for all build products of the project - top-level config/BuildFile.xml - compiler's tools files (e.g. gcc-cxxcompiler.xml, gcc-cxxcompiler.xml and gcc-f77compiler.xml) - via command-line USER_<flag>/SCRAM_<flag>
  • Tool level flags
    Any flags defined in the tool file of an external are used for all the build products which directly or indirectly depends on that tool.
  • Product level flags
    Any flags defined in the BuildFile.xml is used by the product(s) defined in that BuildFile.xml

Some flags (<class>[REM]<flag>) can be configured based on their product class e.g. available classes are LIBRARY For all shared library/edm plugin/edm capabilities plugin products BINARY For all executables. TEST For all test executables. TEST_LIBRARY For all test shared libraires executables.

Some flags (<type>[REM]<flag>) can be configured based on the SCRAM-based area types e.g. available area types are RELEASE Only for compilation/build in the release area environment. DEV Only for compilation/build user development area.

Scram Build Flags

SCRAM sets variuos flags to be used by different compilers (e.g CXX, C and Fortran)

  • [<class|type>][REM]<CXX|C|F>FLAGS
    Compilation flags for CXX/C/Fortran compilers.
  • [<class|type>][REM]CPPFLAGS
    Pre-processor flags for pre-processing.
  • [<class|type>][REM]CPPDEFINES
    Pre-processor macros, SCRAM adds -D for each of these flags.
  • [<class|type>][REM]LDFLAGS
    Link flags for linking shared libraries or building executables.
  • [CXX|C|F]OPTIMISEDFLAGS
    Optiomization flags for CXX/C/Fortran
  • [CXX|C|F]CXXSHAREDOBJECTFLAGS
    Shared object flags for CXX/C/Fortran
  • [REM_]CUDA_FLAGS
    Compilation flags for CUDA compiler.
  • [REM_]CUDA_CFLAGS
    Compilation flags for CUDA compiler which should passes via --compiler-options.
  • [<class>][REM]<EDM|CAPABILITIES>_<CPPFLAGS|CXXFLAGS|CFLAGS>
    Compilation flags for EDM/Capabilities Plugin compilation
  • [<class>][REM]EDM_LDFLAGS
    Link flags for EDM Plugin linking.
  • [REM_]LCGDICT_LDFLAGS
    Link flags for EDM Capabilities Plugin linking.
  • [<class>][REM]<LCG|ROOT>DICT_<CPPFLAGS|CXXFLAGS>
    Compilation flags to compile generated lcg/root dictionary files.
  • LD_UNIT
    Flags used for the generation of big object file for big plugins.
  • MISSING_SYMBOL_FLAGS
    Link flags used for linking to make sure there are no missing symbols.
  • BIGOBJ_[REM_]<CPPFLAGS|CXXFLAGS|CFLAGS|FFLAGS|LDFLAGS>
    Various compilation/link flags for Big Plugins.
  • GENREFLEX_ARGS
    Flags/arguments for genreflex
  • GENREFLEX_GCCXMLOPT
    GCCXML options passed to genreflex
  • GENREFLEX_CPPFLAGS
    Pre-processor flags pass to genreflex

Scram Control Flags

There are few control flags which one can add in to Buildfile.xml to control scram build process e.g.

  • <export><lib="1"/></export>
    This makes a shared library generated form <package>/BuildFile.xml linkable.

  • RIVET_PLUGIN=1|0
    To tell SCRAM to generate a RIVET Plugin instead of a shared library. Default value is 0.

  • EDM_PLUGIN=1|0
    To tell SCRAM to generate a EDM Plugin instead of a shared library. Default value is 1 for <package>/<plugins>/BuildFile.xml and 0 for all otherBuildFile.xml

  • ADD_SUBDIR=1|0
    To tell SCRAM to search for source files in all sub-directories pf <package>/src. Default value is 0 i.e. scram only looks for source files (c, cc, cpp, ccx) in <package>/src directory

  • NO_LIB_CHECKING=1|0
    To tell SCRAM to not check the generated shared library/edm plugin for missing symbols. Default value is 0.

  • LCG_DICT_HEADER=<files>
    Space separated list of classes.h files for LCG dictionaries. Default value is classes.h

  • LCG_DICT_XML=<files>
    Space separated list of classes_def.xml files for LCG dictionaries. Default value is classes_def.xml

  • ROOTMAP=1|0
    Generate .rootmap file too. Default value is 0

  • GENREFLEX_FAILES_ON_WARNS=1|0
    To tell scram to fail on genreflex warnings. Default value is 0.

  • SKIP_FILES=<files>
    Space separated list of source files which should not be considered for compilation.

  • INSTALL_SCRIPTS=<files>
    Space separated list of scripts which should be copied to bin/<arch> product store.

  • NO_TESTRUN=1|0
    To avoid running the unit test via "scram build runtests". This flag is only valid in <package>/test/BuildFile.xml

  • TEST_RUNNER_ARGS=<options>
    Command-line arguments to be passed to the test when test is run via "scram build runtests" This flag is only valid in <package>/test/BuildFile.xml

  • TEST_RUNNER_CMD=<command>
    The command to run the test via "scram build runtests" This flag is only valid in <package>/test/BuildFile.xml

  • USE_UNITTEST_DIR=1
    To run unit tests in <your-dev-area>/unit_tests/<unit-test-name> directory. This flag is only valid in <package>/test/BuildFile.xml

  • SETENV=<env_var>=<value>
    This sets the environment variable before running the unit test. This flag is only valid in <package>/test/BuildFile.xml

  • SETENV_SCRIPT=<scrpit-to-source>
    This sources the script to set environment before running the unit test. This flag is only valid in <package>/test/BuildFile.xml

  • PRE_TEST=<test>
    Tests which should be run first before run this test. This flag is only valid in <package>/test/BuildFile.xml

  • DROP_DEP=<dependency>
    For Big EDM Plugins, to drop any direct or indirect dependnecy used by the packages of Big Plugin.

  • LLVM_PLUGIN
    Name of the static analyzer plugin for LLVM Static Analysis. Effective only in config/BuildFile.xml.

  • LLVM_CHECKERS
    Default checkers to be enbabled for LLVM Static Analysis. Effective only in config/BuildFile.xml.

  • <include_path path="<dir>"/>
    To add -I<dir> during compilation. <dir> could be full path or relative to the BuildFile.xml.

  • <library name="<name>" file="<files>">[dependencies/flags]</library>
    To generate a shared library from a <packages>/[plugins|test|bin] directory.

    By default, SCRAM generates shared library for &lt;package&gt;/src directory so no
need to use &lt;library&gt; tag in &lt;package&gt;/BuildFile.xml.
By default, all shared libraries generated from &lt;packages&gt;/plugins are EMD Plugins.
unless explicitly turned off by &lt;flags EDM_PLUGIN="0"/&gt;

  • <bin name="<name>" file="<files>">[dependencies/flags]</bin>
    To generate an executable/test executable from a <packages>/[plugins|test|bin] directory.

  • <test name="<name>" command="<command-to-run>" [for="[start=1,]end[,step=1]"|foreach="comma,separated,values"] >[dependencies/flags]</test>
    To run the command for the test in <package>/test directory. "for and foreach" are only available in SCRAMV3

    When multiple tests are generated then one can make use of ${value}, ${step_value}, ${start_value} and ${end_value} e.g
**&lt;test name="foo" command="echo Run ${value} of max ${end_value}" for="2,6,2"/&gt;**

All tests are run from project top level directory. Depending on build rules, scram can set few environment variables for each test too e.g.
  SCRAM_TEST_NAME=name of the unit tests e.g MyTest
  SCRAM_TEST_PATH=Full path of the test directory e.g $(LOCALTOP)/src/Subsystem/Package/test
  SCRAM_TEST_PACKAGE=Name of the project package e.g. Subsystem/Package

  • <set name="VARIABLE" value="VALUE"/>
    Only available in SCRAMV3, this sets VARIABLE=VALUE which one can reuse with in the BuildFile. e.g. if multiple unit tests needs a common arg then you can set it once and then use ${VARIABLE} in unit tests commands.

  • <if<condition> name|match="[!]<regexp>"/></if<condition>>
    OR <if<condition> value="[!]<value>"/></if<condition>> OR <if<condition> name|match|value=="[!]<value>"/> <!-- contents --> <elif name|match|value="[!]<value>"/> <!-- contents --> <else/> <!-- contents --> </if<condition>>

Any flag/dependency within these tags will apply if regexp/value match the condition. If first charactor of regexp/value is '!' then flags/dependency will apply if regexp/value doe not match.

Valid conditions are architecture: SCRAM_ARCH environment variable arch: System architecture only e.g. x86_64, ppc64le, aarch64 os: Operating system type e.g. linux, darwin etc. compiler : Default compiler from config/Self.xml release: SCRAM_PROJECTVERSION environment variable project: SCRAM_PROJECTNAME environment variable scram: SCRAM_VERSION environment variable config: SCRAM_CONFIGCHKSUM environment variable cxx11_abi: SCRAM_CXX11_ABI environment variable tool: To check for a tool (and optionally its version) e.g <iftool name="root" version="6..="></iftool>

Nasted conditions blocks are allowed.

CONTROL FLAGS (via environment)

There are few environment flags that can control SCRAM e.g.

  • USER_[BIGOBJ_][REM_]<CPPFLAGS|CXXFLAGS|CFLAGS|FFLAGS|LDFLAGS>
    Various user defined compilation/link flags.
  • USER_LLVM_CHECKERS
    Used defined extra checkers to be enabled for LLVM Static Analysis.
  • USER_SCRAM_TARGET
    To override the default cpu family selection for multi-architecture builds. e.g. if cmssw release is build with extra micro-architecture e.g. "x86-64-v3" then one can use "USER_SCRAM_TARGET=x86-64-v3 cmsenv" to select "x86-64-v3" libs build with "x86-64-v3" instructions set. "USER_SCRAM_TARGET=default" will force select the default instructions set and ""USER_SCRAM_TARGET=auto" will let scram use the best available instructions set. NOTE: If a cmssw release is build with SCRAM_TARGET=default (see config/Self.xml) then "USER_SCRAM_TARGET=x86-64-v3 cmsenv" in release area will properly setup "x86-64-v3" libs but in user development area you need to run "scram build enable-multi-targets" to enable the multi-targets support (by default multi-targets are disabled in user dev area for such releases). Once we build cmssw release with "SCRAM_TARGET=auto" then multi-targets will also be enabled for user dev area and "scram build" in user dev area will build all locally checked out packages for mult-archs. One can disable multi-targets fo such releases by running ""scram build disable-multi-targets.
  • SCRAM_USERLOOKUPDB=<path>
    To instruct SCRAM to use <path> as its database and look for projects under this directory.
  • RUN_LLVM_ANALYZER_ON_ALL=yes
    To run llvm analyzer on generated code too. By default it does not run on generated code.
  • BUILD_LOG=yes
    To redirect the "scram build -j n" output to log files for each product.
  • SCRAM_NOEDMWRITECONFIG=1
    No to run EDM Write Config script after the edmplugin build.
  • SCRAM_IGNORE_PACKAGES=<packages>
    Do not build <packages>
  • SCRAM_IGNORE_SUBDIRS=<sub-dirs>
    Do not build <sub-dirs> of each package e.g. one can set it to test to avoid building test executables/plugins.
  • SKIP_UNITTESTS=<tests>
    Do not run these <tests> when "scram build runtests|unittests" is run.
  • SCRAM_NOSYMCHECK=1
    Do not run any extra shared library missing symbols checks.
  • SCRAM_TEST_RUNNER_PREFIX=<command>
    Prefix each unittest with <command> before running.
  • SCRAM_PREFIX_<path-variable>=<path-string>
    SCRAM runtime command prefixes <path-string> for each <path-variable>. <path-variable> could be PATH, LD_LIBRARY_PATH, PYTHON*PATH.
  • SCRAM_IGNORE_PROJECT_HOOK=1
    Do not run project creation hooks defined by each SCRAM base project in their config/SCRAM/hooks path
  • SCRAM_IGNORE_RUNTIME_HOOK=1
    Do not run runtime environment setup hooks defined by each SCRAM base project in their config/SCRAM/hooks path
  • SCRAM_IGNORE_SITE_PROJECT_HOOK=1
    Do not run project creation hooks defined by site in <InstallPath>/etc/scramrc/SCRAM/hooks
  • SCRAM_IGNORE_SITE_RUNTIME_HOOK=1
    Do not run runtime environment setup hooks defined site in <InstallPath>/etc/scramrc/SCRAM/hooks
  • SCRAM_HOOKS_DEBUG=1
    To get debug output from scram runtime hooks
  • SCRAM_SITE_HOOKS_DIR=/path/to/site/specific/hooks
    Override the SCRAM site-specific hooks paths, by default SCRAM uses <InstallPath>/etc/scramrc to find hooks
  • _SCRAM_TOOL_PATH_HANDLER=1
    Internally used by scram hooks to ignore the missing tool paths e.g. for creating on fly the cuda paths