aws-glue+spark-workshop.ipynb

{
	"metadata": {
		"kernelspec": {
			"name": "glue_pyspark",
			"display_name": "Glue PySpark",
			"language": "python"
		},
		"language_info": {
			"name": "Python_Glue_Session",
			"mimetype": "text/x-python",
			"codemirror_mode": {
				"name": "python",
				"version": 3
			},
			"pygments_lexer": "python3",
			"file_extension": ".py"
		}
	},
	"nbformat_minor": 4,
	"nbformat": 4,
	"cells": [
		{
			"cell_type": "markdown",
			"source": "# AWS Glue y PySpark",
			"metadata": {}
		},
		{
			"cell_type": "markdown",
			"source": "El contenido de este notebook se basa en la documentación de AWS para GLue que se encuentra disponible en:\n\nhttps://github.com/aws-samples/aws-glue-samples/blob/master/examples/join_and_relationalize.md\n\nMagic commands en glue:\n\nhttps://docs.aws.amazon.com/glue/latest/dg/interactive-sessions-magics.html",
			"metadata": {}
		},
		{
			"cell_type": "markdown",
			"source": "### Iniciar la sesión interactiva y el spark context\n",
			"metadata": {
				"editable": true,
				"trusted": true
			}
		},
		{
			"cell_type": "code",
			"source": "%idle_timeout 2880\n%glue_version 4.0\n%worker_type G.1X\n%number_of_workers 2\n\nimport sys\nfrom awsglue.transforms import *\nfrom awsglue.utils import getResolvedOptions\nfrom pyspark.context import SparkContext\nfrom awsglue.context import GlueContext\nfrom awsglue.job import Job\n  \nsc = SparkContext.getOrCreate()\nglueContext = GlueContext(sc)\nspark = glueContext.spark_session\njob = Job(glueContext)",
			"metadata": {
				"trusted": true,
				"editable": true
			},
			"execution_count": 1,
			"outputs": [
				{
					"name": "stdout",
					"text": "Current idle_timeout is None minutes.\nidle_timeout has been set to 2880 minutes.\nSetting Glue version to: 4.0\nPrevious worker type: None\nSetting new worker type to: G.1X\nPrevious number of workers: None\nSetting new number of workers to: 2\nTrying to create a Glue session for the kernel.\nSession Type: glueetl\nWorker Type: G.1X\nNumber of Workers: 2\nIdle Timeout: 2880\nSession ID: db73530b-ead1-49f5-8eb7-1fc1aacdfe0b\nApplying the following default arguments:\n--glue_kernel_version 1.0.5\n--enable-glue-datacatalog true\nWaiting for session db73530b-ead1-49f5-8eb7-1fc1aacdfe0b to get into ready status...\nSession db73530b-ead1-49f5-8eb7-1fc1aacdfe0b has been created.\n\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 1. Creamos el bucket de salida desde la consola de S3\n\ns3://awsglue-datasets/examples/us-legislators\n\n### 2. Copiamos los datos del bucket de origen a un bucket en nuestra cuenta\n\naws s3 sync s3://awsglue-datasets/examples/us-legislators/all s3://glue-sample-target-3404648233655478/examples/us-legislators/all --dryrun\n",
			"metadata": {}
		},
		{
			"cell_type": "markdown",
			"source": "### 3. Creamos un crawler para añadir la información al catálogo de Glue\n\nEl nombre de la base de datos es legislators, al ejecutar el crawler se crean las siguientes tablas:\n\npersons_json\n\nmemberships_json\n\norganizations_json\n\nevents_json\n\nareas_json\n\ncountries_r_json\n",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# catalog: database and table names\ndb_name = \"legislators\"\ntbl_persons = \"persons_json\"\ntbl_membership = \"memberships_json\"\ntbl_organization = \"organizations_json\"\n\n# output s3 and temp directories\noutput_history_dir = \"s3://glue-data-out-3404648233655478/output-dir/legislator_history\"\noutput_lg_single_dir = \"s3://glue-data-out-3404648233655478/output-dir/legislator_single\"\noutput_lg_partitioned_dir = \"s3://glue-data-out-3404648233655478/output-dir/legislator_part\"\nathena_temp_dir = \"s3://glue-data-out-3404648233655478/temp-dir/\"",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 2,
			"outputs": [
				{
					"name": "stdout",
					"text": "\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 4. Procedemos a crear un dynamic_frame a partir de la información que está en el catálogo",
			"metadata": {
				"tags": []
			}
		},
		{
			"cell_type": "code",
			"source": "persons = glueContext.create_dynamic_frame.from_catalog(\n             database=\"legislators\",\n             table_name=\"persons_json\")\nprint (\"Count: \", persons.count())\npersons.printSchema()",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 3,
			"outputs": [
				{
					"name": "stdout",
					"text": "Count:  1961\nroot\n|-- family_name: string\n|-- name: string\n|-- links: array\n|    |-- element: struct\n|    |    |-- note: string\n|    |    |-- url: string\n|-- gender: string\n|-- image: string\n|-- identifiers: array\n|    |-- element: struct\n|    |    |-- scheme: string\n|    |    |-- identifier: string\n|-- other_names: array\n|    |-- element: struct\n|    |    |-- lang: string\n|    |    |-- note: string\n|    |    |-- name: string\n|-- sort_name: string\n|-- images: array\n|    |-- element: struct\n|    |    |-- url: string\n|-- given_name: string\n|-- birth_date: string\n|-- id: string\n|-- contact_details: array\n|    |-- element: struct\n|    |    |-- type: string\n|    |    |-- value: string\n|-- death_date: string\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 5. Leemos las 3 tablas que vamos a procesar",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# Create dynamic frames from the source tables \npersons = glueContext.create_dynamic_frame.from_catalog(database=db_name, table_name=tbl_persons)\nmemberships = glueContext.create_dynamic_frame.from_catalog(database=db_name, table_name=tbl_membership)\norgs = glueContext.create_dynamic_frame.from_catalog(database=db_name, table_name=tbl_organization)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 4,
			"outputs": [
				{
					"name": "stdout",
					"text": "\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "print (\"Count: \", persons.count())\npersons.show(1)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 36,
			"outputs": [
				{
					"name": "stdout",
					"text": "Count:  1961\n{\"family_name\": \"Collins\", \"name\": \"Mac Collins\", \"links\": [{\"note\": \"Wikipedia (de)\", \"url\": \"https://de.wikipedia.org/wiki/Mac_Collins\"}, {\"note\": \"Wikipedia (en)\", \"url\": \"https://en.wikipedia.org/wiki/Mac_Collins\"}, {\"note\": \"Wikipedia (sv)\", \"url\": \"https://sv.wikipedia.org/wiki/Mac_Collins\"}, {\"note\": \"website\", \"url\": \"http://www.house.gov/maccollins\"}], \"gender\": \"male\", \"image\": \"https://theunitedstates.io/images/congress/original/C000640.jpg\", \"identifiers\": [{\"scheme\": \"bioguide\", \"identifier\": \"C000640\"}, {\"scheme\": \"everypolitician_legacy\", \"identifier\": \"C000640\"}, {\"scheme\": \"freebase\", \"identifier\": \"/m/0255g_\"}, {\"scheme\": \"google_entity_id\", \"identifier\": \"kg:/m/0255g_\"}, {\"scheme\": \"govtrack\", \"identifier\": \"400078\"}, {\"scheme\": \"house_history\", \"identifier\": \"11254\"}, {\"scheme\": \"icpsr\", \"identifier\": \"29340\"}, {\"scheme\": \"nndb\", \"identifier\": \"229/000036121\"}, {\"scheme\": \"opensecrets\", \"identifier\": \"N00002556\"}, {\"scheme\": \"snac\", \"identifier\": \"w6086f24\"}, {\"scheme\": \"thomas\", \"identifier\": \"00222\"}, {\"scheme\": \"uscongress\", \"identifier\": \"C000640\"}, {\"scheme\": \"viaf\", \"identifier\": \"258630693\"}, {\"scheme\": \"wikidata\", \"identifier\": \"Q1882459\"}, {\"scheme\": \"wikipedia\", \"identifier\": \"Mac Collins\"}], \"other_names\": [{\"lang\": \"bar\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"ca\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"da\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"de\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"en\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"es\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"fa\", \"note\": \"multilingual\", \"name\": \"مک کالینز\"}, {\"lang\": \"fi\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"fr\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"hu\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"it\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"lb\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"nb\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"nds\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"nl\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"nn\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}, {\"lang\": \"sv\", \"note\": \"multilingual\", \"name\": \"Mac Collins\"}], \"sort_name\": \"Collins, Michael\", \"images\": [{\"url\": \"https://theunitedstates.io/images/congress/original/C000640.jpg\"}, {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/2/26/MacCollins.JPG\"}], \"given_name\": \"Michael\", \"birth_date\": \"1944-10-15\", \"id\": \"0005af3a-9471-4d1f-9299-737fff4b9b46\"}\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "print (\"Count: \", memberships.count())\nmemberships.show(1)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 37,
			"outputs": [
				{
					"name": "stdout",
					"text": "Count:  10439\n{\"area_id\": \"ocd-division/country:us/state:ga/cd:3\", \"on_behalf_of_id\": \"party/republican\", \"organization_id\": \"d56acebe-8fdc-47bc-8b53-20170a3214bc\", \"role\": \"member\", \"person_id\": \"0005af3a-9471-4d1f-9299-737fff4b9b46\", \"legislative_period_id\": \"term/103\"}\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "print (\"Count: \", orgs.count())\norgs.show()",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 41,
			"outputs": [
				{
					"name": "stdout",
					"text": "Count:  13\n{\"classification\": \"party\", \"org_id\": \"party/al\", \"org_name\": \"AL\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.democrats.org/\"}], \"image\": \"https://upload.wikimedia.org/wikipedia/commons/7/73/US_Democratic_Party_Logo.svg\", \"org_id\": \"party/democrat\", \"org_name\": \"Democrat\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.liberalparty.org/\"}], \"org_id\": \"party/democrat-liberal\", \"org_name\": \"Democrat-Liberal\"}\n{\"classification\": \"legislature\", \"seats\": 435, \"type\": \"lower house\", \"org_id\": \"d56acebe-8fdc-47bc-8b53-20170a3214bc\", \"org_name\": \"House of Representatives\"}\n{\"classification\": \"party\", \"org_id\": \"party/independent\", \"org_name\": \"Independent\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.pnppr.com\"}], \"image\": \"https://upload.wikimedia.org/wikipedia/commons/7/75/PNP.logo.png\", \"org_id\": \"party/new_progressive\", \"org_name\": \"New Progressive\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.ppdpr.net/\"}], \"org_id\": \"party/popular_democrat\", \"org_name\": \"Popular Democrat\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.gop.com/\"}], \"image\": \"https://upload.wikimedia.org/wikipedia/commons/d/d7/Logo-GOP.svg\", \"org_id\": \"party/republican\", \"org_name\": \"Republican\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.cpnys.org/\"}], \"org_id\": \"party/republican-conservative\", \"org_name\": \"Republican-Conservative\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.democrats.org/\"}], \"image\": \"https://upload.wikimedia.org/wikipedia/commons/7/73/US_Democratic_Party_Logo.svg\", \"org_id\": \"party/democrat\", \"org_name\": \"Democrat\"}\n{\"classification\": \"party\", \"org_id\": \"party/independent\", \"org_name\": \"Independent\"}\n{\"classification\": \"party\", \"links\": [{\"note\": \"website\", \"url\": \"http://www.gop.com/\"}], \"image\": \"https://upload.wikimedia.org/wikipedia/commons/d/d7/Logo-GOP.svg\", \"org_id\": \"party/republican\", \"org_name\": \"Republican\"}\n{\"classification\": \"legislature\", \"seats\": 100, \"type\": \"upper house\", \"org_id\": \"8fa6c3d2-71dc-4788-b9f8-4ca70d5a7d85\", \"org_name\": \"Senate\"}\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 6. Eliminamos algunos campos innecesarios",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# Keep the fields we need and rename some.\norgs = orgs.drop_fields(['other_names', 'identifiers']).rename_field('id', 'org_id').rename_field('name', 'org_name')\n",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 5,
			"outputs": [
				{
					"name": "stdout",
					"text": "\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 7. Unimos la información de los conjuntos de datos",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# Join the frames to create history\nl_history = Join.apply(orgs, Join.apply(persons, memberships, 'id', 'person_id'), 'org_id', 'organization_id').drop_fields(['person_id', 'org_id'])",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 6,
			"outputs": [
				{
					"name": "stdout",
					"text": "\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "l_history.printSchema()",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 7,
			"outputs": [
				{
					"name": "stdout",
					"text": "root\n|-- role: string\n|-- seats: int\n|-- org_name: string\n|-- links: array\n|    |-- element: struct\n|    |    |-- note: string\n|    |    |-- url: string\n|-- type: string\n|-- sort_name: string\n|-- area_id: string\n|-- images: array\n|    |-- element: struct\n|    |    |-- url: string\n|-- on_behalf_of_id: string\n|-- other_names: array\n|    |-- element: struct\n|    |    |-- lang: string\n|    |    |-- note: string\n|    |    |-- name: string\n|-- name: string\n|-- birth_date: string\n|-- organization_id: string\n|-- gender: string\n|-- classification: string\n|-- death_date: string\n|-- legislative_period_id: string\n|-- identifiers: array\n|    |-- element: struct\n|    |    |-- scheme: string\n|    |    |-- identifier: string\n|-- image: string\n|-- given_name: string\n|-- start_date: string\n|-- family_name: string\n|-- id: string\n|-- contact_details: array\n|    |-- element: struct\n|    |    |-- type: string\n|    |    |-- value: string\n|-- end_date: string\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 8. Almacenamos los resultados en formato parquet",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# ---- Write out the history ----\n\n# Write out the dynamic frame into parquet in \"legislator_history\" directory\nprint(\"Writing to /legislator_history ...\")\nglueContext.write_dynamic_frame.from_options(frame = l_history, connection_type = \"s3\", connection_options = {\"path\": output_history_dir}, format = \"parquet\")\n",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 8,
			"outputs": [
				{
					"name": "stdout",
					"text": "Writing to /legislator_history ...\n<awsglue.dynamicframe.DynamicFrame object at 0x7fb6151316c0>\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 9. Reparticionamos el dataset para tener un único archivo",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "# Write out a single file to directory \"legislator_single\"\ns_history = l_history.toDF().repartition(1)\nprint(\"Writing to /legislator_single ...\")\ns_history.write.parquet(output_lg_single_dir)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 10,
			"outputs": [
				{
					"name": "stdout",
					"text": "Writing to /legislator_single ...\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 10. Particionamos el archivo por los campos Senate and House y \"Aplanamos los datos\" para quitar los campos anidados",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "\n# Convert to data frame, write to directory \"legislator_part\", partitioned by (separate) Senate and House.\nprint(\"Writing to /legislator_part, partitioned by Senate and House ...\")\nl_history.toDF().write.parquet(output_lg_partitioned_dir, partitionBy=['org_name'])\n\n# ---- Write out to relational databases ----\n\n# Convert the data to flat tables\nprint(\"Converting to flat tables ...\")\ndfc = l_history.relationalize(\"hist_root\", athena_temp_dir)\n",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 11,
			"outputs": [
				{
					"name": "stdout",
					"text": "Writing to /legislator_part, partitioned by Senate and House ...\nConverting to flat tables ...\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "dyf_out = dfc.select(\"hist_root\")\n\ndyf_out.printSchema()\n\ndf_final = dyf_out.toDF()",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 14,
			"outputs": [
				{
					"name": "stdout",
					"text": "root\n|-- role: string\n|-- seats: int\n|-- org_name: string\n|-- links: long\n|-- type: string\n|-- sort_name: string\n|-- area_id: string\n|-- images: long\n|-- on_behalf_of_id: string\n|-- other_names: long\n|-- birth_date: string\n|-- name: string\n|-- organization_id: string\n|-- gender: string\n|-- classification: string\n|-- death_date: string\n|-- identifiers: long\n|-- legislative_period_id: string\n|-- given_name: string\n|-- image: string\n|-- start_date: string\n|-- family_name: string\n|-- id: string\n|-- contact_details: long\n|-- end_date: string\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "df_final.show(n=1, truncate=False, vertical=True)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 17,
			"outputs": [
				{
					"name": "stdout",
					"text": "-RECORD 0--------------------------------------------------------------------------------\n role                  | member                                                          \n seats                 | 435                                                             \n org_name              | House of Representatives                                        \n links                 | 1                                                               \n type                  | lower house                                                     \n sort_name             | Nagle, David                                                    \n area_id               | ocd-division/country:us/state:ia/cd:3                           \n images                | 1                                                               \n on_behalf_of_id       | party/democrat                                                  \n other_names           | 1                                                               \n birth_date            | 1943-04-15                                                      \n name                  | David Nagle                                                     \n organization_id       | d56acebe-8fdc-47bc-8b53-20170a3214bc                            \n gender                | male                                                            \n classification        | legislature                                                     \n death_date            | null                                                            \n identifiers           | 1                                                               \n legislative_period_id | term/100                                                        \n given_name            | David                                                           \n image                 | https://theunitedstates.io/images/congress/original/N000003.jpg \n start_date            | null                                                            \n family_name           | Nagle                                                           \n id                    | 28f68685-555d-425c-9794-9c904e3864e7                            \n contact_details       | null                                                            \n end_date              | null                                                            \nonly showing top 1 row\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "markdown",
			"source": "### 11. Consultar datos usando comandos SQL",
			"metadata": {}
		},
		{
			"cell_type": "code",
			"source": "df_final.createOrReplaceTempView(\"legislator_table\")",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 19,
			"outputs": [
				{
					"name": "stdout",
					"text": "\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "spark.sql(\"SELECT COUNT(*) n_rows FROM legislator_table\").show(n=50, truncate=False)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 42,
			"outputs": [
				{
					"name": "stdout",
					"text": "+------+\n|n_rows|\n+------+\n|10439 |\n+------+\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "spark.sql(\"SELECT DISTINCT type,org_name,seats FROM legislator_table\").show(n=50, truncate=False)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 45,
			"outputs": [
				{
					"name": "stdout",
					"text": "+-----------+------------------------+-----+\n|type       |org_name                |seats|\n+-----------+------------------------+-----+\n|lower house|House of Representatives|435  |\n|upper house|Senate                  |100  |\n+-----------+------------------------+-----+\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "spark.sql(\"SELECT DISTINCT sort_name, classification, type, org_name FROM legislator_table\").show(n=100, truncate=False)",
			"metadata": {
				"trusted": true,
				"tags": []
			},
			"execution_count": 43,
			"outputs": [
				{
					"name": "stdout",
					"text": "+--------------------------+--------------+-----------+------------------------+-----+\n|sort_name                 |classification|type       |org_name                |seats|\n+--------------------------+--------------+-----------+------------------------+-----+\n|Danielson, George         |legislature   |lower house|House of Representatives|435  |\n|Franks, Bob               |legislature   |lower house|House of Representatives|435  |\n|Herseth Sandlin, Stephanie|legislature   |lower house|House of Representatives|435  |\n|Gowdy, Trey               |legislature   |lower house|House of Representatives|435  |\n|Meehan, Patrick           |legislature   |lower house|House of Representatives|435  |\n|Young, Don                |legislature   |lower house|House of Representatives|435  |\n|Anthony, Beryl            |legislature   |lower house|House of Representatives|435  |\n|Stivers, Steve            |legislature   |lower house|House of Representatives|435  |\n|Hawkins, Augustus         |legislature   |lower house|House of Representatives|435  |\n|Carney, William           |legislature   |lower house|House of Representatives|435  |\n|Teague, Harry             |legislature   |lower house|House of Representatives|435  |\n|Markey, Edward            |legislature   |lower house|House of Representatives|435  |\n|Conte, Silvio             |legislature   |lower house|House of Representatives|435  |\n|Wasserman Schultz, Debbie |legislature   |lower house|House of Representatives|435  |\n|Farr, Sam                 |legislature   |lower house|House of Representatives|435  |\n|Roe, David                |legislature   |lower house|House of Representatives|435  |\n|Mahoney, Tim              |legislature   |lower house|House of Representatives|435  |\n|Donnelly, Brian           |legislature   |lower house|House of Representatives|435  |\n|Harper, Gregg             |legislature   |lower house|House of Representatives|435  |\n|Derrick, Butler           |legislature   |lower house|House of Representatives|435  |\n|Dingell, John             |legislature   |lower house|House of Representatives|435  |\n|McEachin, A.              |legislature   |lower house|House of Representatives|435  |\n|Panetta, Jimmy            |legislature   |lower house|House of Representatives|435  |\n|Zimmer, Richard           |legislature   |lower house|House of Representatives|435  |\n|Schaffer, Bob             |legislature   |lower house|House of Representatives|435  |\n|Moolenaar, John           |legislature   |lower house|House of Representatives|435  |\n|Patman, William           |legislature   |lower house|House of Representatives|435  |\n|Weller, Gerald            |legislature   |lower house|House of Representatives|435  |\n|Reed, John                |legislature   |lower house|House of Representatives|435  |\n|Parker, Michael           |legislature   |lower house|House of Representatives|435  |\n|Curbelo, Carlos           |legislature   |lower house|House of Representatives|435  |\n|Blumenauer, Earl          |legislature   |lower house|House of Representatives|435  |\n|Baker, Richard            |legislature   |lower house|House of Representatives|435  |\n|Price, David              |legislature   |lower house|House of Representatives|435  |\n|Bereuter, Douglas         |legislature   |lower house|House of Representatives|435  |\n|Wenstrup, Brad            |legislature   |lower house|House of Representatives|435  |\n|Seastrand, Andrea         |legislature   |lower house|House of Representatives|435  |\n|Smith, Lawrence           |legislature   |lower house|House of Representatives|435  |\n|Mitchell, Donald          |legislature   |lower house|House of Representatives|435  |\n|Lawson, Al                |legislature   |lower house|House of Representatives|435  |\n|McCarthy, Carolyn         |legislature   |lower house|House of Representatives|435  |\n|Reed, Tom                 |legislature   |lower house|House of Representatives|435  |\n|Atkins, Chester           |legislature   |lower house|House of Representatives|435  |\n|Titus, Dina               |legislature   |lower house|House of Representatives|435  |\n|Brady, Robert             |legislature   |lower house|House of Representatives|435  |\n|Petri, Thomas             |legislature   |lower house|House of Representatives|435  |\n|Vela, Filemon             |legislature   |lower house|House of Representatives|435  |\n|Leach, James              |legislature   |lower house|House of Representatives|435  |\n|Kramer, Kenneth           |legislature   |lower house|House of Representatives|435  |\n|Hollenbeck, Harold        |legislature   |lower house|House of Representatives|435  |\n|Bentivolio, Kerry         |legislature   |lower house|House of Representatives|435  |\n|Guthrie, Brett            |legislature   |lower house|House of Representatives|435  |\n|Harris, Katherine         |legislature   |lower house|House of Representatives|435  |\n|LaFalce, John             |legislature   |lower house|House of Representatives|435  |\n|Ackerman, Gary            |legislature   |lower house|House of Representatives|435  |\n|Walsh, James              |legislature   |lower house|House of Representatives|435  |\n|McCarthy, Kevin           |legislature   |lower house|House of Representatives|435  |\n|Knollenberg, Joseph       |legislature   |lower house|House of Representatives|435  |\n|DeSaulnier, Mark          |legislature   |lower house|House of Representatives|435  |\n|Broyhill, James           |legislature   |lower house|House of Representatives|435  |\n|Alexander, William        |legislature   |lower house|House of Representatives|435  |\n|Gianforte, Greg           |legislature   |lower house|House of Representatives|435  |\n|Israel, Steve             |legislature   |lower house|House of Representatives|435  |\n|Ratcliffe, John           |legislature   |lower house|House of Representatives|435  |\n|Coffman, Mike             |legislature   |lower house|House of Representatives|435  |\n|Kazen, Abraham            |legislature   |lower house|House of Representatives|435  |\n|Patterson, Jerry          |legislature   |lower house|House of Representatives|435  |\n|Gaydos, Joseph            |legislature   |lower house|House of Representatives|435  |\n|Gottheimer, Josh          |legislature   |lower house|House of Representatives|435  |\n|Jones, Walter             |legislature   |lower house|House of Representatives|435  |\n|Murphy, Scott             |legislature   |lower house|House of Representatives|435  |\n|Michel, Robert            |legislature   |lower house|House of Representatives|435  |\n|Clayton, Eva              |legislature   |lower house|House of Representatives|435  |\n|Dold, Bob                 |legislature   |lower house|House of Representatives|435  |\n|Clawson, Curtis           |legislature   |lower house|House of Representatives|435  |\n|Sensenbrenner, F.         |legislature   |lower house|House of Representatives|435  |\n|Evans, David              |legislature   |lower house|House of Representatives|435  |\n|Lowery, William           |legislature   |lower house|House of Representatives|435  |\n|Garamendi, John           |legislature   |lower house|House of Representatives|435  |\n|Luján, Ben                |legislature   |lower house|House of Representatives|435  |\n|Miller, Candice           |legislature   |lower house|House of Representatives|435  |\n|Thomas, Craig             |legislature   |lower house|House of Representatives|435  |\n|Hodes, Paul               |legislature   |lower house|House of Representatives|435  |\n|Wise, Robert              |legislature   |lower house|House of Representatives|435  |\n|Brown, Sherrod            |legislature   |lower house|House of Representatives|435  |\n|Smith, Joseph             |legislature   |lower house|House of Representatives|435  |\n|Womack, Steve             |legislature   |lower house|House of Representatives|435  |\n|Erlenborn, John           |legislature   |lower house|House of Representatives|435  |\n|Cheney, Liz               |legislature   |lower house|House of Representatives|435  |\n|Cantwell, Maria           |legislature   |lower house|House of Representatives|435  |\n|Lloyd, Marilyn            |legislature   |lower house|House of Representatives|435  |\n|Volkmer, Harold           |legislature   |lower house|House of Representatives|435  |\n|Boucher, Frederick        |legislature   |lower house|House of Representatives|435  |\n|Brady, Kevin              |legislature   |lower house|House of Representatives|435  |\n|Cardin, Benjamin          |legislature   |lower house|House of Representatives|435  |\n|Rudd, Eldon               |legislature   |lower house|House of Representatives|435  |\n|Berman, Howard            |legislature   |lower house|House of Representatives|435  |\n|Mooney, Alex              |legislature   |lower house|House of Representatives|435  |\n|Funderburk, David         |legislature   |lower house|House of Representatives|435  |\n|Tonko, Paul               |legislature   |lower house|House of Representatives|435  |\n+--------------------------+--------------+-----------+------------------------+-----+\nonly showing top 100 rows\n",
					"output_type": "stream"
				}
			]
		},
		{
			"cell_type": "code",
			"source": "",
			"metadata": {},
			"execution_count": null,
			"outputs": []
		}
	]
}