5. Use Case - Working with AWS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![title](http://berytech.org/wp-content/uploads/2015/11/amazon-aws-logo.jpg)\n",
    "\n",
    "# Use Case - Working with AWS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Working with AWS\n",
    "\n",
    "- AWS has a great API towards python\n",
    "- Almost everything you can do in the GUI can be done through this interface, like managing servers, s3 buckets etc.\n",
    "- The API library is called [boto3](https://boto3.readthedocs.io/en/latest/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Setting up boto3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download and install AWS CLI. This can actually be done through the Python Package Index:\n",
    "\n",
    "    pip install awscli"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tp configure AWS CLI, run the following command:\n",
    "\n",
    "    aws configure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Importing the stuff we need"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import boto3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Setting up the s3 client\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "client = boto3.client('s3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Listing buckets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'CreationDate': datetime.datetime(2016, 10, 29, 15, 50, 16, tzinfo=tzutc()),\n",
       " 'Name': 'adm-lju-backupbucket1'}"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.list_buckets()['Buckets'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Listing files in a bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ETag': '\"d41d8cd98f00b204e9800998ecf8427e\"',\n",
       " 'Key': 'bootstrapping/',\n",
       " 'LastModified': datetime.datetime(2017, 1, 26, 8, 59, 5, tzinfo=tzutc()),\n",
       " 'Owner': {'DisplayName': 'vlpaws+503375299761',\n",
       "  'ID': 'b80fe58644fc3d9801d3e0c72e1698a48bfdc327af7fbaafed98e8a190d7e375'},\n",
       " 'Size': 0,\n",
       " 'StorageClass': 'STANDARD'}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bucket_name = 'adm-psa-testbucket'\n",
    "client.list_objects(Bucket=bucket_name)['Contents'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Downloading a file, modifying it, and writing it back"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ETag': '\"e4257ed235b628835b4a0c271c25822a\"',\n",
       " 'ResponseMetadata': {'HTTPHeaders': {'content-length': '0',\n",
       "   'date': 'Thu, 16 Mar 2017 15:00:30 GMT',\n",
       "   'etag': '\"e4257ed235b628835b4a0c271c25822a\"',\n",
       "   'server': 'AmazonS3',\n",
       "   'x-amz-id-2': '7Q3SYGSMLrumwfI1WlXwc1r90peghfuZzS9Rbo5tE8Z0Quk0ppidxq5cejxy7GH7Rae5BlbTPtM=',\n",
       "   'x-amz-request-id': '02AA91C87A5C05F3'},\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HostId': '7Q3SYGSMLrumwfI1WlXwc1r90peghfuZzS9Rbo5tE8Z0Quk0ppidxq5cejxy7GH7Rae5BlbTPtM=',\n",
       "  'RequestId': '02AA91C87A5C05F3',\n",
       "  'RetryAttempts': 0}}"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The bucket name\n",
    "bucket_name = 'adm-psa-testbucket'\n",
    "\n",
    "# Path to the file\n",
    "key = 'python-presentation/test.txt'\n",
    "\n",
    "# Get the file object, and read the contant\n",
    "obj = client.get_object(Bucket=bucket_name, Key=key)\n",
    "text = str(obj['Body'].read(), 'utf-8')\n",
    "\n",
    "# Number each of the lines in the file text\n",
    "lines = [line.strip() for line in text.split(\"\\n\")]\n",
    "new_text = \"\\r\\n\".join(\"{}. {}\".format(i, line) for i, line in enumerate(lines))\n",
    "\n",
    "# Write the updated file back to s3\n",
    "new_key = 'python-presentation/test_modified.txt'\n",
    "client.put_object(Bucket=bucket_name, Key=new_key, Body=bytes(new_text, 'utf-8'))"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}