Merge pull request #1132 from awslabs/develop

v0.15.0 Release

.gitignore

@@ -380,5 +380,9 @@ $RECYCLE.BIN/
 
 # Temporary scratch directory used by the tests
 tests/integration/buildcmd/scratch
+tests/integration/testdata/buildcmd/Dotnetcore2.0/bin
+tests/integration/testdata/buildcmd/Dotnetcore2.0/obj
+tests/integration/testdata/buildcmd/Dotnetcore2.1/bin
+tests/integration/testdata/buildcmd/Dotnetcore2.1/obj
 
 # End of https://www.gitignore.io/api/osx,node,macos,linux,python,windows,pycharm,intellij,sublimetext,visualstudiocode

.travis.yml

@@ -25,6 +25,15 @@ before_install:
   - nvm install 8.10
   - npm --version
   - node --version
+  # Install .NET Core 2.1
+  - export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 DOTNET_CLI_TELEMETRY_OPTOUT=1
+  - if [ "$LINUX" ]; then sudo apt install libunwind8; fi
+  - wget https://dot.net/v1/dotnet-install.sh -O /tmp/dotnet-install.sh
+  - chmod +x /tmp/dotnet-install.sh
+  - /tmp/dotnet-install.sh -v 2.1.504
+  - export DOTNET_ROOT=/home/travis/.dotnet
+  - export PATH=/home/travis/.dotnet:/home/travis/.dotnet/tools:$PATH
+  - dotnet --info
 
 install:
   # Install the code requirements

README.md

@@ -7,10 +7,12 @@ SAM CLI (Beta)
 ![Build
 Status](https://travis-ci.org/awslabs/aws-sam-cli.svg?branch=develop)
 ![Apache-2.0](https://img.shields.io/npm/l/aws-sam-local.svg)
-![Contributers](https://img.shields.io/github/contributors/awslabs/aws-sam-cli.svg)
+![Contributors](https://img.shields.io/github/contributors/awslabs/aws-sam-cli.svg)
 ![GitHub-release](https://img.shields.io/github/release/awslabs/aws-sam-cli.svg)
 ![PyPI version](https://badge.fury.io/py/aws-sam-cli.svg)
 
+[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-black.svg)](https://snapcraft.io/aws-sam-cli)
+
 [Join the SAM developers channel (\#samdev) on
 Slack](https://join.slack.com/t/awsdevelopers/shared_invite/enQtMzg3NTc5OTM2MzcxLTdjYTdhYWE3OTQyYTU4Njk1ZWY4Y2ZjYjBhMTUxNGYzNDg5MWQ1ZTc5MTRlOGY0OTI4NTdlZTMwNmI5YTgwOGM/)
 to collaborate with fellow community members and the AWS SAM team.

designs/build_debug_artifacts.md

@@ -0,0 +1,135 @@
+SAM Build producing debuggable artifacts
+========================================
+
+What is the problem?
+--------------------
+
+`sam build` produces artifacts that are built for production use. In some langauges (usually interpreted), 
+these production artifacts are also debuggable locally. In the case of compiled languages, you usually need to compile
+the binary or artifact in a specific manner for them to be debuggable.  
+
+What will be changed?
+---------------------
+
+We will introduce a way in `sam build` to produce these debuggable artifacts for those compiled languages.
+
+Success criteria for the change
+-------------------------------
+
+1. Artifacts generated will be debuggable for runtimes DotNetCore 2.0 and above. 
+
+Out-of-Scope
+------------
+
+1. Other languages `sam build` supports will not be changed
+
+User Experience Walkthrough
+---------------------------
+
+Implementation
+==============
+
+CLI Changes
+-----------
+
+*Explain the changes to command line interface, including adding new
+commands, modifying arguments etc*
+
+### Breaking Change
+
+Changes are additive and will not break any existing functionality.
+
+Design
+------
+
+Options considered
+------------------
+We have a couple options to consider:
+
+1. A command line option (`--mode`)
+    * Pros
+        * Customer can control what kind of artifacts are produced.
+        * There is no guessing needed by the CLI
+    * Cons
+        * Yet another option for customers to learn or know about
+        * Makes the customer need to think about the artifacts they need to produce (though this can be hidden
+        behind the AWS Toolkit in IDEs)
+        * Customers running in the command line need to remember what artifacts to produce or what they previously produced
+2. An Environment Variable we read (`SAM_BUILD_MODE`). IDE Toolkit will set the env var when calling `sam build`
+while debugging.
+    * Pros
+        * Reduces cognitive load on customers that don't care about debugging dotnet apps through command line.
+        * Could force customers to use AWS Toolkit by default
+    * Cons
+        * Building debug artifacts becomes a hidden feature (silent contract)
+        * Environment Variables tend to be more set and forget.
+        * Need to conditionally add Env Var instead of a more convenient flag
+3. Seamless Integration: `sam build` produces debug artifacts by default and `sam package` will build 
+non debug artifacts by default
+    * Pros
+        * Customers should never have to think about 'when to produce debug artifacts' or forget to add a flag
+    * Cons
+        * Requires additional work on package to auto build.
+        * Breaks what is produced from `sam build` as build is positioned to produce artifacts that are ready for deployment
+
+Proposal
+--------
+
+My recommendation is to follow Option #2 from above, mainly because:
+
+- Seamless Integration (#3) is best experience but is large in scope.
+- CLI Option (#1) exposes a flag for what we think will be a rarely used CLI feature. We think so because manually 
+setting up debugging is cumbersome. Given that .NET developers generally prefer to be within the IDE, I find it hard 
+to believe someone will want to go out of the way of setting it up through CLI.
+- Not a one-way-door. We can always add a CLI option to pair with env var later.
+
+`.samrc` Changes
+----------------
+
+*Explain the new configuration entries, if any, you want to add to
+.samrc*
+
+Security
+--------
+
+*Tip: How does this change impact security? Answer the following
+questions to help answer this question better:*
+
+**What new dependencies (libraries/cli) does this change require?**
+No
+
+**What other Docker container images are you using?**
+N/A
+
+**Are you creating a new HTTP endpoint? If so explain how it will be
+created & used**
+No
+
+**Are you connecting to a remote API? If so explain how is this
+connection secured**
+No
+
+**Are you reading/writing to a temporary folder? If so, what is this
+used for and when do you clean up?**
+No
+
+**How do you validate new .samrc configuration?**
+N/A
+
+Documentation Changes
+---------------------
+
+Open Issues
+-----------
+
+Task Breakdown
+--------------
+
+-   \[x\] Send a Pull Request with this design document
+-   \[ \] Build the command line interface
+-   \[ \] Build the underlying library
+-   \[ \] Unit tests
+-   \[ \] Functional Tests
+-   \[ \] Integration tests
+-   \[ \] Run all tests on Windows
+-   \[ \] Update documentation

designs/sam_build_cmd.md

@@ -64,6 +64,8 @@ Success criteria for the change
 8.  Integrate build action with `sam local/package/deploy` commands so
     the Lambda functions will be automatically built as part of the
     command without explicitly running the build command.
+9.  Support for building the app for debugging locally with debug
+    symbols (ex: Golang) [see design doc](build_debug_artifacts.md)
 
 Out-of-Scope
 ------------
@@ -76,8 +78,6 @@ Out-of-Scope
     package manager (ex: images, css etc)
 3.  Support to exclude certain files from the built artifact (ex: using
     .gitignore or using regex)
-4.  Support for building the app for debugging locally with debug
-    symbols (ex: Golang)
 5.  Support caching dependencies & re-installing them only when the
     dependency manifest changes (ex: by maintaining hash of
     package.json)

requirements/base.txt

@@ -12,5 +12,5 @@ dateparser~=0.7
 python-dateutil~=2.6
 pathlib2~=2.3.2; python_version<"3.4"
 requests==2.20.1
-aws_lambda_builders==0.2.1
 serverlessrepo==0.1.8
+aws_lambda_builders==0.3.0

samcli/__init__.py

@@ -2,4 +2,4 @@
 SAM CLI version
 """
 
-__version__ = '0.14.2'
+__version__ = '0.15.0'

samcli/commands/build/build_context.py

@@ -26,6 +26,7 @@ def __init__(self,
                  template_file,
                  base_dir,
                  build_dir,
+                 mode,
                  manifest_path=None,
                  clean=False,
                  use_container=False,
@@ -42,6 +43,7 @@ def __init__(self,
         self._parameter_overrides = parameter_overrides
         self._docker_network = docker_network
         self._skip_pull_image = skip_pull_image
+        self._mode = mode
 
         self._function_provider = None
         self._template_dict = None
@@ -73,21 +75,16 @@ def __exit__(self, *args):
 
     @staticmethod
     def _setup_build_dir(build_dir, clean):
+        build_path = pathlib.Path(build_dir)
 
-        # Get absolute path
-        build_dir = str(pathlib.Path(build_dir).resolve())
-
-        if not pathlib.Path(build_dir).exists():
-            # Build directory does not exist. Create the directory and all intermediate paths
-            os.makedirs(build_dir, BuildContext._BUILD_DIR_PERMISSIONS)
-
-        if os.listdir(build_dir) and clean:
-            # Build folder contains something inside. Clear everything.
+        if build_path.exists() and os.listdir(build_dir) and clean:
+            # build folder contains something inside. Clear everything.
             shutil.rmtree(build_dir)
-            # this would have cleared the parent folder as well. So recreate it.
-            os.mkdir(build_dir, BuildContext._BUILD_DIR_PERMISSIONS)
 
-        return build_dir
+        build_path.mkdir(mode=BuildContext._BUILD_DIR_PERMISSIONS, parents=True, exist_ok=True)
+
+        # ensure path resolving is done after creation: https://bugs.python.org/issue32434
+        return str(build_path.resolve())
 
     @property
     def container_manager(self):
@@ -127,3 +124,7 @@ def manifest_path_override(self):
             return os.path.abspath(self._manifest_path)
 
         return None
+
+    @property
+    def mode(self):
+        return self._mode

samcli/commands/build/command.py

@@ -11,7 +11,8 @@
 from samcli.commands._utils.options import template_option_without_build, docker_common_options, \
     parameter_override_option
 from samcli.commands.build.build_context import BuildContext
-from samcli.lib.build.app_builder import ApplicationBuilder, BuildError, UnsupportedBuilderLibraryVersionError
+from samcli.lib.build.app_builder import ApplicationBuilder, BuildError, UnsupportedBuilderLibraryVersionError, \
+    ContainerBuildNotSupported
 from samcli.lib.build.workflow_config import UnsupportedRuntimeException
 from samcli.commands._utils.template import move_template
 
@@ -31,9 +32,10 @@
 Supported Runtimes
 ------------------
 1. Python 2.7, 3.6, 3.7 using PIP\n
-4. Nodejs 8.10, 6.10 using NPM
-4. Ruby 2.5 using Bundler
-5. Java 8 using Gradle
+2. Nodejs 8.10, 6.10 using NPM\n
+3. Ruby 2.5 using Bundler\n
+4. Java 8 using Gradle\n
+5. Dotnetcore2.0 and 2.1 using Dotnet CLI (without --use-container flag)\n
 \b
 Examples
 --------
@@ -88,11 +90,14 @@ def cli(ctx,
         manifest,
         docker_network,
         skip_pull_image,
-        parameter_overrides):
+        parameter_overrides,
+        ):
     # All logic must be implemented in the ``do_cli`` method. This helps with easy unit testing
 
+    mode = _get_mode_value_from_envvar("SAM_BUILD_MODE", choices=["debug"])
+
     do_cli(template, base_dir, build_dir, True, use_container, manifest, docker_network,
-           skip_pull_image, parameter_overrides)  # pragma: no cover
+           skip_pull_image, parameter_overrides, mode)  # pragma: no cover
 
 
 def do_cli(template,  # pylint: disable=too-many-locals
@@ -103,7 +108,8 @@ def do_cli(template,  # pylint: disable=too-many-locals
            manifest_path,
            docker_network,
            skip_pull_image,
-           parameter_overrides):
+           parameter_overrides,
+           mode):
     """
     Implementation of the ``cli`` method
     """
@@ -121,13 +127,15 @@ def do_cli(template,  # pylint: disable=too-many-locals
                       use_container=use_container,
                       parameter_overrides=parameter_overrides,
                       docker_network=docker_network,
-                      skip_pull_image=skip_pull_image) as ctx:
+                      skip_pull_image=skip_pull_image,
+                      mode=mode) as ctx:
 
         builder = ApplicationBuilder(ctx.function_provider,
                                      ctx.build_dir,
                                      ctx.base_dir,
                                      manifest_path_override=ctx.manifest_path_override,
-                                     container_manager=ctx.container_manager
+                                     container_manager=ctx.container_manager,
+                                     mode=ctx.mode
                                      )
         try:
             artifacts = builder.build()
@@ -147,8 +155,9 @@ def do_cli(template,  # pylint: disable=too-many-locals
 
             click.secho(msg, fg="yellow")
 
-        except (UnsupportedRuntimeException, BuildError, UnsupportedBuilderLibraryVersionError) as ex:
-            click.secho("Build Failed", fg="red")
+        except (UnsupportedRuntimeException, BuildError, UnsupportedBuilderLibraryVersionError,
+                ContainerBuildNotSupported) as ex:
+            click.secho("\nBuild Failed", fg="red")
             raise UserException(str(ex))
 
 
@@ -175,3 +184,16 @@ def gen_success_msg(artifacts_dir, output_template_path, is_default_build_dir):
                template=output_template_path)
 
     return msg
+
+
+def _get_mode_value_from_envvar(name, choices):
+
+    mode = os.environ.get(name, None)
+    if not mode:
+        return None
+
+    if mode not in choices:
+        raise click.UsageError("Invalid value for 'mode': invalid choice: {}. (choose from {})"
+                               .format(mode, choices))
+
+    return mode