00_prune_preprocess.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "average-division",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Pruning-the-SNPs\" data-toc-modified-id=\"Pruning-the-SNPs-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Pruning the SNPs</a></span><ul class=\"toc-item\"><li><span><a href=\"#Extract-the-required-SNPs-per-phenotype\" data-toc-modified-id=\"Extract-the-required-SNPs-per-phenotype-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Extract the required SNPs per phenotype</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cleared-gnome",
   "metadata": {},
   "source": [
    "This notebook will show how to download the Supplementary in Lippert et al 2013, with their resulting datasets. In the Excel file below there will be two different sets of SNPs calculated from the WTCCC, corresponding to the results of their analyses with the standard method or with the expanded method. \n",
    "\n",
    "The expanded includes as controls other cases for the other diseases, which increases the number of samples & power, so we will also compare against this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "intensive-cable",
   "metadata": {},
   "outputs": [],
   "source": [
    "wget https://static-content.springer.com/esm/art%3A10.1038%2Fsrep01099/MediaObjects/41598_2013_BFsrep01099_MOESM3_ESM.xls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "animal-prescription",
   "metadata": {},
   "source": [
    "Save as txt those files..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "considered-america",
   "metadata": {},
   "outputs": [],
   "source": [
    "snpset=\"test_lippert\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "american-pixel",
   "metadata": {},
   "outputs": [],
   "source": [
    "cd ~/Projects/DeepCOMBI\n",
    "mkdir -p ${snpset}\n",
    "mkdir -p ${snpset}\"_pruning\";\n",
    "cd ${snpset}\"_pruning\";"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "threatened-percentage",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/jrodriguez/Projects/DeepCOMBI/test_lippert_pruning\n"
     ]
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "designing-drilling",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Phenotype\tSNP\tChr\tPosition\tPvalue\t>1 Pheno\tnot LMM\n",
      "bd\trs1442650\t18\t39970159\t6,13833E-11\t\t\n",
      "cd\trs10210302\t2\t234158839\t8,59582E-09\t\t\n",
      "cd\trs10213846\t5\t40442869\t4,95444E-10\t\t\n",
      "cd\trs10512734\t5\t40393605\t3,73626E-08\t\t\n",
      "cd\trs10789224\t1\t67605134\t1,92257E-07\t\t\n",
      "cd\trs11747270\t5\t150258867\t1,28539E-07\t\t\n",
      "cd\trs11805303\t1\t67675516\t2,48649E-07\t\t\n",
      "cd\trs11957134\t5\t150230950\t3,41151E-07\t\t\n",
      "cd\trs11957215\t5\t40445681\t5,18105E-10\t\t\n"
     ]
    }
   ],
   "source": [
    "head ../${snpset}.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "located-management",
   "metadata": {},
   "source": [
    "# Pruning the SNPs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prescribed-desert",
   "metadata": {},
   "source": [
    "## Extract the required SNPs per phenotype\n",
    "In hg19"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fitting-perth",
   "metadata": {},
   "outputs": [],
   "source": [
    "sed 1d ../${snpset}.txt | cut -f1 | sort | uniq > phenos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "thirty-symphony",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bd\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.1.bim: No such file or directory\n",
      "Error: Failed to open bd.1.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.2.bim: No such file or directory\n",
      "Error: Failed to open bd.2.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.3.bim: No such file or directory\n",
      "Error: Failed to open bd.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.4.bim: No such file or directory\n",
      "Error: Failed to open bd.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.5.bim: No such file or directory\n",
      "Error: Failed to open bd.5.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.6.bim: No such file or directory\n",
      "Error: Failed to open bd.6.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.7.bim: No such file or directory\n",
      "Error: Failed to open bd.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.8.bim: No such file or directory\n",
      "Error: Failed to open bd.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.9.bim: No such file or directory\n",
      "Error: Failed to open bd.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.10.bim: No such file or directory\n",
      "Error: Failed to open bd.10.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.11.bim: No such file or directory\n",
      "Error: Failed to open bd.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.12.bim: No such file or directory\n",
      "Error: Failed to open bd.12.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.13.bim: No such file or directory\n",
      "Error: Failed to open bd.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.14.bim: No such file or directory\n",
      "Error: Failed to open bd.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.15.bim: No such file or directory\n",
      "Error: Failed to open bd.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.16.bim: No such file or directory\n",
      "Error: Failed to open bd.16.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.17.bim: No such file or directory\n",
      "Error: Failed to open bd.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.18.bim: No such file or directory\n",
      "Error: Failed to open bd.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.19.bim: No such file or directory\n",
      "Error: Failed to open bd.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.20.bim: No such file or directory\n",
      "Error: Failed to open bd.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.21.bim: No such file or directory\n",
      "Error: Failed to open bd.21.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: bd.22.bim: No such file or directory\n",
      "Error: Failed to open bd.22.bed.\n",
      "cd\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.3.bim: No such file or directory\n",
      "Error: Failed to open cd.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.4.bim: No such file or directory\n",
      "Error: Failed to open cd.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.6.bim: No such file or directory\n",
      "Error: Failed to open cd.6.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.7.bim: No such file or directory\n",
      "Error: Failed to open cd.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.8.bim: No such file or directory\n",
      "Error: Failed to open cd.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.9.bim: No such file or directory\n",
      "Error: Failed to open cd.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.10.bim: No such file or directory\n",
      "Error: Failed to open cd.10.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.11.bim: No such file or directory\n",
      "Error: Failed to open cd.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.12.bim: No such file or directory\n",
      "Error: Failed to open cd.12.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.13.bim: No such file or directory\n",
      "Error: Failed to open cd.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.14.bim: No such file or directory\n",
      "Error: Failed to open cd.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.15.bim: No such file or directory\n",
      "Error: Failed to open cd.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.17.bim: No such file or directory\n",
      "Error: Failed to open cd.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.18.bim: No such file or directory\n",
      "Error: Failed to open cd.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.19.bim: No such file or directory\n",
      "Error: Failed to open cd.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.20.bim: No such file or directory\n",
      "Error: Failed to open cd.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.21.bim: No such file or directory\n",
      "Error: Failed to open cd.21.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: cd.22.bim: No such file or directory\n",
      "Error: Failed to open cd.22.bed.\n",
      "ht\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.1.bim: No such file or directory\n",
      "Error: Failed to open ht.1.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.2.bim: No such file or directory\n",
      "Error: Failed to open ht.2.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.3.bim: No such file or directory\n",
      "Error: Failed to open ht.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.4.bim: No such file or directory\n",
      "Error: Failed to open ht.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.5.bim: No such file or directory\n",
      "Error: Failed to open ht.5.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.6.bim: No such file or directory\n",
      "Error: Failed to open ht.6.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.7.bim: No such file or directory\n",
      "Error: Failed to open ht.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.8.bim: No such file or directory\n",
      "Error: Failed to open ht.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.9.bim: No such file or directory\n",
      "Error: Failed to open ht.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.10.bim: No such file or directory\n",
      "Error: Failed to open ht.10.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.11.bim: No such file or directory\n",
      "Error: Failed to open ht.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.12.bim: No such file or directory\n",
      "Error: Failed to open ht.12.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.13.bim: No such file or directory\n",
      "Error: Failed to open ht.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.14.bim: No such file or directory\n",
      "Error: Failed to open ht.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.15.bim: No such file or directory\n",
      "Error: Failed to open ht.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.16.bim: No such file or directory\n",
      "Error: Failed to open ht.16.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.17.bim: No such file or directory\n",
      "Error: Failed to open ht.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.18.bim: No such file or directory\n",
      "Error: Failed to open ht.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.19.bim: No such file or directory\n",
      "Error: Failed to open ht.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.20.bim: No such file or directory\n",
      "Error: Failed to open ht.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ht.21.bim: No such file or directory\n",
      "Error: Failed to open ht.21.bed.\n",
      "ra\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.2.bim: No such file or directory\n",
      "Error: Failed to open ra.2.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.3.bim: No such file or directory\n",
      "Error: Failed to open ra.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.4.bim: No such file or directory\n",
      "Error: Failed to open ra.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.5.bim: No such file or directory\n",
      "Error: Failed to open ra.5.bed.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.7.bim: No such file or directory\n",
      "Error: Failed to open ra.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.8.bim: No such file or directory\n",
      "Error: Failed to open ra.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.9.bim: No such file or directory\n",
      "Error: Failed to open ra.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.11.bim: No such file or directory\n",
      "Error: Failed to open ra.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.12.bim: No such file or directory\n",
      "Error: Failed to open ra.12.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.13.bim: No such file or directory\n",
      "Error: Failed to open ra.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.14.bim: No such file or directory\n",
      "Error: Failed to open ra.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.15.bim: No such file or directory\n",
      "Error: Failed to open ra.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.16.bim: No such file or directory\n",
      "Error: Failed to open ra.16.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.17.bim: No such file or directory\n",
      "Error: Failed to open ra.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.18.bim: No such file or directory\n",
      "Error: Failed to open ra.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.19.bim: No such file or directory\n",
      "Error: Failed to open ra.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.20.bim: No such file or directory\n",
      "Error: Failed to open ra.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.21.bim: No such file or directory\n",
      "Error: Failed to open ra.21.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: ra.22.bim: No such file or directory\n",
      "Error: Failed to open ra.22.bed.\n",
      "t1d\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.2.bim: No such file or directory\n",
      "Error: Failed to open t1d.2.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.3.bim: No such file or directory\n",
      "Error: Failed to open t1d.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.4.bim: No such file or directory\n",
      "Error: Failed to open t1d.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.5.bim: No such file or directory\n",
      "Error: Failed to open t1d.5.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.7.bim: No such file or directory\n",
      "Error: Failed to open t1d.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.8.bim: No such file or directory\n",
      "Error: Failed to open t1d.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.9.bim: No such file or directory\n",
      "Error: Failed to open t1d.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.10.bim: No such file or directory\n",
      "Error: Failed to open t1d.10.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.11.bim: No such file or directory\n",
      "Error: Failed to open t1d.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.13.bim: No such file or directory\n",
      "Error: Failed to open t1d.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.14.bim: No such file or directory\n",
      "Error: Failed to open t1d.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.15.bim: No such file or directory\n",
      "Error: Failed to open t1d.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.16.bim: No such file or directory\n",
      "Error: Failed to open t1d.16.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.17.bim: No such file or directory\n",
      "Error: Failed to open t1d.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.18.bim: No such file or directory\n",
      "Error: Failed to open t1d.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.19.bim: No such file or directory\n",
      "Error: Failed to open t1d.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.20.bim: No such file or directory\n",
      "Error: Failed to open t1d.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.21.bim: No such file or directory\n",
      "Error: Failed to open t1d.21.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t1d.22.bim: No such file or directory\n",
      "Error: Failed to open t1d.22.bed.\n",
      "t2d\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.1.bim: No such file or directory\n",
      "Error: Failed to open t2d.1.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.2.bim: No such file or directory\n",
      "Error: Failed to open t2d.2.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.3.bim: No such file or directory\n",
      "Error: Failed to open t2d.3.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.4.bim: No such file or directory\n",
      "Error: Failed to open t2d.4.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.5.bim: No such file or directory\n",
      "Error: Failed to open t2d.5.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.6.bim: No such file or directory\n",
      "Error: Failed to open t2d.6.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.7.bim: No such file or directory\n",
      "Error: Failed to open t2d.7.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.8.bim: No such file or directory\n",
      "Error: Failed to open t2d.8.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.9.bim: No such file or directory\n",
      "Error: Failed to open t2d.9.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.11.bim: No such file or directory\n",
      "Error: Failed to open t2d.11.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.13.bim: No such file or directory\n",
      "Error: Failed to open t2d.13.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.14.bim: No such file or directory\n",
      "Error: Failed to open t2d.14.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.15.bim: No such file or directory\n",
      "Error: Failed to open t2d.15.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.16.bim: No such file or directory\n",
      "Error: Failed to open t2d.16.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.17.bim: No such file or directory\n",
      "Error: Failed to open t2d.17.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.18.bim: No such file or directory\n",
      "Error: Failed to open t2d.18.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.19.bim: No such file or directory\n",
      "Error: Failed to open t2d.19.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.20.bim: No such file or directory\n",
      "Error: Failed to open t2d.20.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.21.bim: No such file or directory\n",
      "Error: Failed to open t2d.21.bed.\n",
      "Error: All variants excluded by '--extract range'.\n",
      "bash: t2d.22.bim: No such file or directory\n",
      "Error: Failed to open t2d.22.bed.\n"
     ]
    },
    {
     "ename": "",
     "evalue": "2",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "while read p; do \n",
    "    echo $p\n",
    "    awk -v OFS='\\t' -v enf=${p} '{if($1 == enf){print $0}}' ../${snpset}.txt > ${p}.snps\n",
    "    for chr in {1..22}; do\n",
    "        awk -v c=${chr} '{if($3 == c){print $3,$4,$4,$2}}' ${p}.snps > ${p}.${chr}.range;\n",
    "        plink --bfile ~/scratch/chr_files_1KG_CEU_MAF001/ALL.chr${chr}_CEU_MAF001 --silent --extract range ${p}.${chr}.range --keep-allele-order --make-bed --out ${p}.${chr}        \n",
    "        lins=$(wc -l < ${p}.${chr}.bim)                \n",
    "        if [[ \"$lins\" == 1 ]];\n",
    "        then\n",
    "            awk -v OFS='\\t' '{print $2}' ${p}.${chr}.bim > ${p}.${chr}.prune.in\n",
    "        else\n",
    "            ## SNP sliding\n",
    "            # plink --bfile ${p}.${chr} --silent --indep-pairwise 2 1 0.8 --out ${p}.${chr}\n",
    "            \n",
    "            ## Prune on distance\n",
    "            plink --bfile ${p}.${chr} --silent --indep-pairwise 500 kb 1 0.2 --out ${p}.${chr}\n",
    "        fi;\n",
    "    done\n",
    "done < phenos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "eligible-quantity",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rm: cannot remove ‘all.bd.txt’: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.2.prune.in: No such file or directory\n",
      "paste: bd.2.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.3.prune.in: No such file or directory\n",
      "paste: bd.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.4.prune.in: No such file or directory\n",
      "paste: bd.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.5.prune.in: No such file or directory\n",
      "paste: bd.5.prune.inhead: option requires an argument -- 'n'\n",
      ": No such file or directory\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.7.prune.in: No such file or directory\n",
      "paste: bd.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.8.prune.in: No such file or directory\n",
      "paste: bd.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "wc: bd.9.prune.inyes: standard output: No such file or directory\n",
      ": Broken pipe\n",
      "yes: write error\n",
      "paste: bd.9.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.10.prune.in: No such file or directory\n",
      "paste: bd.10.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.11.prune.in: No such file or directory\n",
      "paste: bd.11.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.12.prune.in: No such file or directory\n",
      "paste: bd.12.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.13.prune.in: No such file or directory\n",
      "paste: bd.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "paste: bd.14.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.15.prune.in: No such file or directory\n",
      "paste: bd.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.16.prune.in: No such file or directory\n",
      "paste: bd.16.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.17.prune.in: No such file or directory\n",
      "paste: bd.17.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "wc: yes: bd.18.prune.instandard output: Broken pipe\n",
      "yes: write error\n",
      ": No such file or directory\n",
      "paste: bd.18.prune.inhead: option requires an argument -- 'n'\n",
      ": No such file or directory\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.19.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: bd.19.prune.in: No such file or directory\n",
      "standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.20.prune.in: No such file or directory\n",
      "paste: bd.20.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.21.prune.in: No such file or directory\n",
      "paste: bd.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: bd.22.prune.in: No such file or directory\n",
      "paste: bd.22.prune.inhead: option requires an argument -- 'n'\n",
      ": No such file or directory\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.cad.txt’: No such file or directory\n",
      "wc: cad.1.prune.in: No such file or directory\n",
      "paste: cad.1.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.2.prune.in: No such file or directory\n",
      "paste: cad.2.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.3.prune.in: No such file or directory\n",
      "paste: cad.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.4.prune.in: No such file or directory\n",
      "paste: cad.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard outputwc: cad.5.prune.in: Broken pipe\n",
      "yes: write error\n",
      ": No such file or directory\n",
      "paste: cad.5.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.7.prune.in: No such file or directory\n",
      "paste: cad.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.8.prune.in: No such file or directory\n",
      "paste: cad.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.10.prune.in: No such file or directory\n",
      "paste: cad.10.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.11.prune.in: No such file or directory\n",
      "paste: cad.11.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.12.prune.in: No such file or directory\n",
      "paste: cad.12.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.13.prune.in: No such file or directory\n",
      "paste: cad.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.14.prune.in: No such file or directory\n",
      "paste: cad.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.16.prune.in: No such file or directory\n",
      "paste: cad.16.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.17.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "cad.17.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.18.prune.in: No such file or directory\n",
      "paste: cad.18.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.19.prune.in: No such file or directory\n",
      "paste: cad.19.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.20.prune.in: No such file or directory\n",
      "paste: cad.20.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.21.prune.in: No such file or directory\n",
      "paste: cad.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cad.22.prune.in: No such file or directory\n",
      "paste: cad.22.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.cd.txt’: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.4.prune.in: No such file or directory\n",
      "paste: cd.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.7.prune.in: No such file or directory\n",
      "paste: cd.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.8.prune.in: No such file or directory\n",
      "paste: cd.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.9.prune.in: No such file or directory\n",
      "paste: cd.9.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.11.prune.in: No such file or directory\n",
      "paste: cd.11.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.12.prune.in: No such file or directory\n",
      "paste: cd.12.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.13.prune.in: No such file or directory\n",
      "paste: cd.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "wc: Try 'head --help' for more information.\n",
      "cd.14.prune.in: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: cd.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "wc: cd.15.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: cd.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.17.prune.in: No such file or directory\n",
      "paste: cd.17.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.20.prune.in: No such file or directory\n",
      "paste: cd.20.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.21.prune.in: No such file or directory\n",
      "paste: cd.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: cd.22.prune.in: No such file or directory\n",
      "paste: cd.22.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.ht.txt’: No such file or directory\n",
      "wc: ht.1.prune.in: No such file or directory\n",
      "paste: ht.1.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.3.prune.in: No such file or directory\n",
      "paste: ht.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.4.prune.in: No such file or directory\n",
      "paste: ht.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.5.prune.in: No such file or directory\n",
      "paste: ht.5.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.7.prune.in: No such file or directory\n",
      "paste: ht.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.8.prune.in: No such file or directory\n",
      "paste: ht.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "wc: ht.9.prune.in: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "head: option requires an argument -- 'n'\n",
      "paste: ht.9.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.10.prune.in: No such file or directory\n",
      "paste: ht.10.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.12.prune.in: No such file or directory\n",
      "paste: ht.12.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.13.prune.in: No such file or directory\n",
      "paste: ht.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.15.prune.in: No such file or directory\n",
      "paste: ht.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.16.prune.in: No such file or directory\n",
      "paste: ht.16.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.17.prune.in: No such file or directory\n",
      "paste: ht.17.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.18.prune.in: No such file or directory\n",
      "paste: ht.18.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.19.prune.in: No such file or directory\n",
      "paste: ht.19.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.20.prune.in: No such file or directory\n",
      "paste: ht.20.prune.in: No such file or directory\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ht.21.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: ht.21.prune.in: No such file or directorystandard output: Broken pipe\n",
      "\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.ra.txt’: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.2.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "paste: ra.2.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.3.prune.in: No such file or directory\n",
      "paste: ra.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.4.prune.in: No such file or directory\n",
      "paste: ra.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.5.prune.in: No such file or directory\n",
      "paste: ra.5.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.7.prune.in: No such file or directory\n",
      "paste: ra.7.prune.in: No such file or directory\n",
      "wc: ra.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: ra.8.prune.inhead: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.9.prune.in: No such file or directory\n",
      "paste: ra.9.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.10.prune.in: No such file or directory\n",
      "paste: ra.10.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.11.prune.in: No such file or directory\n",
      "paste: ra.11.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.12.prune.in: No such file or directory\n",
      "paste: ra.12.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.13.prune.in: No such file or directory\n",
      "paste: ra.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.14.prune.in: No such file or directory\n",
      "paste: ra.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.15.prune.in: No such file or directory\n",
      "paste: ra.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.16.prune.in: No such file or directory\n",
      "paste: ra.16.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.17.prune.in: No such file or directory\n",
      "paste: ra.17.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.18.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: ra.18.prune.in: No such file or directorystandard output: Broken pipe\n",
      "yes: write error\n",
      "\n",
      "wc: ra.19.prune.in: No such file or directory\n",
      "paste: ra.19.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.20.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: ra.20.prune.in: No such file or directory\n",
      "wc: ra.21.prune.in: No such file or directory\n",
      "paste: ra.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: ra.22.prune.in: No such file or directory\n",
      "paste: ra.22.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.t1d.txt’: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.2.prune.in: No such file or directory\n",
      "paste: t1d.2.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.3.prune.in: No such file or directory\n",
      "paste: t1d.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.4.prune.in: No such file or directory\n",
      "paste: t1d.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.5.prune.in: No such file or directory\n",
      "paste: t1d.5.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.7.prune.in: No such file or directory\n",
      "paste: t1d.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "wc: t1d.8.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: t1d.8.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.9.prune.in: No such file or directory\n",
      "paste: t1d.9.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.10.prune.in: No such file or directory\n",
      "paste: t1d.10.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.11.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "t1d.11.prune.in: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.13.prune.in: No such file or directory\n",
      "paste: t1d.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.14.prune.in: No such file or directory\n",
      "paste: t1d.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.15.prune.in: No such file or directory\n",
      "paste: t1d.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.18.prune.in: No such file or directory\n",
      "paste: t1d.18.prune.in: No such file or directory\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.19.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "paste: t1d.19.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.20.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "t1d.20.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t1d.21.prune.in: No such file or directory\n",
      "paste: t1d.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "wc: Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "t1d.22.prune.in: No such file or directory\n",
      "paste: t1d.22.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "rm: cannot remove ‘all.t2d.txt’: No such file or directory\n",
      "wc: t2d.1.prune.in: No such file or directory\n",
      "paste: t2d.1.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.3.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "paste: t2d.3.prune.in: No such file or directory\n",
      "wc: t2d.4.prune.in: No such file or directory\n",
      "paste: t2d.4.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.5.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "t2d.5.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.7.prune.in: No such file or directory\n",
      "paste: t2d.7.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.8.prune.in: No such file or directory\n",
      "paste: head: option requires an argument -- 'n'\n",
      "t2d.8.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.9.prune.in: No such file or directory\n",
      "paste: t2d.9.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.11.prune.in: No such file or directory\n",
      "paste: t2d.11.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.13.prune.in: No such file or directory\n",
      "paste: t2d.13.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.14.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "paste: t2d.14.prune.in: No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.15.prune.in: No such file or directory\n",
      "paste: t2d.15.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.17.prune.in: No such file or directory\n",
      "paste: t2d.17.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.18.prune.in: No such file or directory\n",
      "paste: t2d.18.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.19.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "paste: t2d.19.prune.inTry 'head --help' for more information.\n",
      ": No such file or directory\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.20.prune.in: No such file or directory\n",
      "paste: t2d.20.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.21.prune.in: No such file or directory\n",
      "paste: t2d.21.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n",
      "wc: t2d.22.prune.in: No such file or directory\n",
      "paste: t2d.22.prune.in: No such file or directory\n",
      "head: option requires an argument -- 'n'\n",
      "Try 'head --help' for more information.\n",
      "yes: standard output: Broken pipe\n",
      "yes: write error\n"
     ]
    }
   ],
   "source": [
    "while read p; do\n",
    "    rm all.${p}.txt\n",
    "    for chr in {1..22}; do\n",
    "        entries=$(wc -l ${p}.${chr}.prune.in | cut -f1 -d' ')\n",
    "        paste <(yes ${chr} | head -n${entries}) ${p}.${chr}.prune.in >> all.${p}.txt\n",
    "    done\n",
    "    if [ -s all.${p}.txt ]; \n",
    "    then\n",
    "        cp all.${p}.txt ../${snpset};\n",
    "    fi;\n",
    "done < phenos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bacterial-tourism",
   "metadata": {},
   "outputs": [],
   "source": [
    "#./combi_check_SNPs_1.sh \"rheumatoid arthritits\" \"ra\"\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"Crohn's disease\" \"cd\"\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"type II diabetes mellitus\" t2d\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"type I diabetes mellitus\" t1d\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"hypertension\" ht\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"bipolar disorder\" \"bd\"\n",
    "\n",
    "#./combi_check_SNPs_1.sh \"coronary heart disease\" \"cad\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "instructional-juice",
   "metadata": {},
   "outputs": [],
   "source": [
    "# CUANTOS\n",
    "dis=\"cd\"; cat <(cut -f3 ${dis}/ld/ld_candidates_${dis} | sed 1d | sort | uniq ) ${dis}/ld/direct_candidates_${dis} | sort | uniq | wc -l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "threaded-logging",
   "metadata": {},
   "outputs": [],
   "source": [
    "# COMMAND TO GET INDEPENDENT REGIONS\n",
    "cd ~/Projects/DeepCOMBI/std_lippert_vs_GWASCat_July2020/;\n",
    "dis=\"cd\"; cat <(cut -f1,2,3 ${dis}/ld/ld_candidates_${dis} | sed 1d | sort | uniq)  <(cut -f12,13,22 ${dis}/ld/direct_candidates_${dis}.assoc | sed 1d | sort -nk1 -nk2) | sort -nk1 -nk2 | uniq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fundamental-porter",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "global-typing",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acceptable-legislature",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "metallic-beverage",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "educated-bermuda",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "warming-battery",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}