Scrape deduped profiles

[phonedude]

The current (undocumented) API that we currently access returns separate documents that the user may have clustered together. For example, currently for 2001 we have each permutation of the OAI-PMH document, whereas most of these permutations are rolled up into: https://scholar.google.com/citations?view_op=view_citation&hl=en&user=oWQaPnwAAAAJ&alert_preview_top_rm=2&citation_for_view=oWQaPnwAAAAJ:u5HHmVD_uO8C

In short, the list of articles we get from:

https://scholar.google.com/citations?hl=en&user=oWQaPnwAAAAJ

and

https://scholar.google.com/citations?hl=en&user=oWQaPnwAAAAJ&view_op=list_works&sortby=pubdate&cstart=0001&pagesize=100

are not the same.

2001

    H Van de Sompel, C Lagoze, The Open Archives Initiative Protocol for Metadata Harvesting. Protocol Version 1.1, 2 July, .

    ML Nelson, K Maly, Smart objects and open archives, D-Lib Magazine 7 (2), 2001.

    X Liu, K Maly, M Zubair, ML Nelson, Arc: an OAI service provider for cross-archive searching, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, 65-66, 2001.

    Open Archives Initiative, Open Archives Initiative Website, .

    ML Nelson, A survey of complex object technologies for digital libraries, .

    ZM Muela-Meza, Odas a las bibliotecas públicas, La Polilla: Publicación Mensual de la Biblioteca Nacional José Martí, 2001.

    ML Nelson, G Marchionini, G Geisler, M Yang, A bucket architecture for the open video project, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries …, 2001.

    ML Nelson, Better interoperability through the open archives initiative, New review of information networking 7 (1), 133-145, 2001.

    X Liu, K Maly, M Zubair, ML Nelson, Arc-an OAI service provider for digital library federation, D-Lib magazine 7 (4), 2001.

    H Suleman, M Nelson, Multiple Metadata/Best Metadata Return, .

    ML Nelson, B Argue, M Efron, S Denn, MC Pattuelli, A Survey of Complex for Digital Libraries, .

    G Geisler, G Marchionini, M Nelson, R Spinks, M Yang, Interface concepts for the open video project, Proceedings of the Annual Meeting-American Society for Information Science …, 2001.

    OA Initiative–OAI, 2015, .

    C Veure Lagoze, H Van de Sompel, The Open Archives Iniciative Protocol for Metadata Harvesting, Open Archive Iniciative, January, 2001.

    ML Nelson, Buckets: A new digital library technology for preserving NASA research, Journal of Government Information 28 (4), 369-394, 2001.

    Open Archives Initiative, Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH), 2005-05-20]. http://www. openarchives. brig, 2001.

    ML Nelson, NASA/TM-2001-211049 Buckets: Smart Objects for Digital Libraries, .

    OPEN Archives Initiative, Disponível em:< http://www. openarchives. org/index. html>, Acesso em 3, 2001.

    Open Archives Initiative, Open Archives Initiative Frequently Asked Questions, .

    ML Nelson, Buckets: smart objects for digital libraries, Old Dominion University, 2001.

    C Lagoze, HV Sompel, M Nelson, S Warner, The Open Archives Initiative Protocol for Metadata Harvesting http://www. openarchives. org, OAI/openarchivesprotocol. html, 2001.

    H Van de Sompel, C Lagoze, The Open Archives Initiative Protocol for Metadata Harvesting: Protocol Version 1.0, Document Version 2001-01-21, Technical report. Ithaca, NY: Cornell University, 2001. http://www …, 2001.

    H Van de Sompel, C Lagoze, The Open Archives Initiative Protocol for Metadata Harvesting. Open Archives Initiative, 2001, .