Journal coverage report - SJR titles.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cb04b51f",
   "metadata": {},
   "source": [
    "# Report: Library coverage of SJR titles\n",
    "\n",
    "#### Description\n",
    "This notebook allows you to compare list(s) of journal titles from __[Scimago Journal Ranking reports (SJR)](https://www.scimagojr.com/)__ against your library's holdings of journals in Primo. It will generate a spreadsheet of the full-text availability for all titles in the report, automatically!\n",
    "\n",
    "#### Dependencies & requirements\n",
    "This notebook queries the Alma link resolver, so this will only work for libraries that use Primo.\n",
    "\n",
    "To run this notebook, you will need:\n",
    "* Python 3\n",
    "* Jupyter Notebooks\n",
    "* The open source packages that are loaded below\n",
    "\n",
    "#### Notes & disclaimer\n",
    "This code may not be perfect, so it is worth double checking the results. Errors will also be introduced based on the metadata quality:\n",
    "* The ISSNs provided by the SJR report may not match up with the ISSNs in your MARC records\n",
    "* The coverage availability statements are pulled from the link resolver, which are only as good as your electronic records in Alma\n",
    "\n",
    "I am always welcome to collaboration -- if this work can be improved, please reach out!\n",
    "\n",
    "#### Author\n",
    "Created by Roger Reka and last updated 9 February 2023.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae201a1a",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98d24dbc",
   "metadata": {},
   "source": [
    "### Base URL\n",
    "The first thing you need to do to setup this notebook is to identify the base URL for your Alma link resolver. It should look something like this `https://ca01.alma.exlibrisgroup.com/view/uresolver/01UTON_UW/openurl?`.\n",
    "\n",
    "Once you have it, go to the `config.py` file and enter it in the `base_URL` field, in between the quotation marks.\n",
    "\n",
    "You only have to do this once."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8980d423",
   "metadata": {},
   "source": [
    "### Install the required packages\n",
    "If you don't have these installed already, you will need to do this now. This notebook uses several open source Python packages that you will have to install into your environment.\n",
    "\n",
    "* `pandas`\n",
    "* `requests`\n",
    "* `xml.etree.ElementTree`\n",
    "* `re`\n",
    "* `glob`\n",
    "\n",
    "You only have to do this once (aside from updates)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "765e6428",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20537ce9",
   "metadata": {},
   "source": [
    "## Analysis\n",
    "This is the start of the actual report analysis. \n",
    "\n",
    "### Grab your SJR files\n",
    "Go to __[Scimago Journal Ranking reports (SJR)](https://www.scimagojr.com/journalrank.php)__ webpage and download the ranking lists that you are interested in. \n",
    "\n",
    "Place these files in the `/data` folder."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a466911b",
   "metadata": {},
   "source": [
    "### Run the code\n",
    "From this section onwards, you can run all the cells below until the report is generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fbbdb87",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the required Python packages. Note, you will have to install these if you have not yet done this before.\n",
    "\n",
    "import pandas as pd\n",
    "import requests\n",
    "import xml.etree.ElementTree as ElementTree\n",
    "import re\n",
    "import glob\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9b4bd2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the functions from the associated Python file\n",
    "\n",
    "import autoCollectionsFunctions as cf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2dc8b7e2",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Find all the CSV files in the directory\n",
    "\n",
    "files = glob.glob('data/*.csv')\n",
    "files"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7de78772",
   "metadata": {},
   "source": [
    "### Prepare the data\n",
    "This section will prepare the data for querying by combining all the data together into one dataframe, and identifying one ISSN for use in querying the link resolver."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be2121b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an empty dataframe\n",
    "df_journals = pd.DataFrame()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ea50101",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For every csv file, grab the data from the named columns (only first X rows) and append them to the df_all dataframe. \n",
    "# Edit the 'nrows' value to select how many rows from each file should be included. Default is the first 50.\n",
    "\n",
    "for file in files:\n",
    "    df_temp = pd.read_csv(file, sep=';', usecols=['Rank', 'Title', 'Type', 'Issn', 'SJR'], nrows=50)\n",
    "    # Also, add the name of the file to each row\n",
    "    df_temp['Category'] = file\n",
    "    df_journals = pd.concat([df_journals, df_temp], sort=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff2520b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For those rows with multiple ISSNS (indicated with a comma), delete the second ISSN\n",
    "\n",
    "df_journals['q_issn'] = df_journals['Issn'].str.split(',').str[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e5420cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select only the columns we need (remove the original ISSN column)\n",
    "\n",
    "df_journals = df_journals[['Rank', 'Title', 'Type', 'q_issn', 'SJR', 'Category']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f4409a23",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_journals"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13de3293",
   "metadata": {},
   "source": [
    "### Query the link resolver\n",
    "In this section, the notebook will now check every row of the dataframe above and query the ISSN against the Alma link resolver. The link resolver will return a response via structured XML, and the code will parse out the relevant coverage data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f3de1b8",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Search the Open URL link resolver to find the coverage for these journals\n",
    "\n",
    "df_journals[['availability', 'coverage']] = df_journals.apply(cf.searchOpenURL ,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5174a8b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update the availability statements based on the coverage dates (emabargo, not to the present)\n",
    "\n",
    "df_journals[['availability']] = df_journals.apply(cf.coverageStatement_availParser,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68b1590c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_journals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc74844b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Summary\n",
    "\n",
    "df_journals.availability.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08fdd6b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export the results into a CSV file\n",
    "\n",
    "df_journals.to_csv('results/SJR_rankings_report_{}.csv'.format(datetime.datetime.now().strftime(\"%Y_%m_%d_%H%M%S\")))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}