Skip to content

Home

Jump to bottom
Daniel Mietchen edited this page Jul 12, 2021 · 34 revisions

License

Content in this wiki is available under the terms of the CC0 1.0 Universal Public Domain Dedication (CC0 1.0).

Tex bibliography

cat > hellocitation.tex <<EOF
\documentclass{article}
\usepackage[utf8]{inputenc}
\title{Hello, Citation}
\author{Finn Årup Nielsen}
\begin{document}
\maketitle
Hello, Citation \cite{Q21133507},
and \cite{10.1186/S13321-016-0161-3}.
\bibliographystyle{unsrt}
\bibliography{hellocitation}
\end{document}
EOF

pdflatex hellocitation.tex
python -m scholia.tex write-bib-from-aux hellocitation.aux
bibtex hellocitation
pdflatex hellocitation.tex
pdflatex hellocitation.tex

Scraping

Example with NIPS papers

python3 -m scholia.scrape.nips scrape-paper-urls-from-proceedings-url --output=nips2016.txt https://papers.nips.cc/book/advances-in-neural-information-processing-systems-28-2015
cat nips2016.txt | head -10 | xargs -n 1  python -m scholia.scrape.nips paper-url-to-quickstatements

Scholia link

You can get a Scholia link on the left panel in Wikidata if you have an account there. A Javascript file, say User:Fnielsen/scholia-link.js, may be setup:

( function( mw, $ ) {
	// Partially from https://www.wikidata.org/wiki/User:Ls1g/recoin-core.js
    'use strict';
    
	console.log('scholia-link loaded');

	// Magnus Manske's check for item
	// https://www.wikidata.org/wiki/User:Magnus_Manske/missing_props.js
    if ( mw.config.get('wgNamespaceNumber') != 0 ) return ;
	if ( mw.config.get('wgAction') != 'view' ) return ;
	var entityID = mw.config.get( 'wbEntityId' );
	
	function init() 
	{
        $('#p-tb').find('ul').append('<li><a href="https://scholia.toolforge.org/' + entityID + '">Scholia</a></li>')
	}	
		
		
	$( function () {
		mw.hook( 'wikipage.content' ).add( init );
	});

	} ( mediaWiki, jQuery ) );

The Javascript file can be included by adding a line in your commons.js file, e.g., User:Fnielsen/common.js:

// importScript( 'User:Fnielsen/scholia-link.js' );  // Said to be an old format
mw.loader.load('//www.wikidata.org/w/index.php?title=User:Fnielsen/scholia-link.js&action=raw&ctype=text/javascript');

Status

Weblog statistics

The following line takes a considerable time and should likely be submitted to the grid as per https://wikitech.wikimedia.org/wiki/Help:Toolforge/Rules - rather than run on the interactive prompt.

egrep "^\[pid:" uwsgi.log | grep -v "GET /scholia/static/" | grep -v "GET /scholia/images/" | grep -v "HEAD /scholia" | awk '{print $12 " " $13 " " $14 " " $15 " " $16 " " $18 }' > stats-`date -I`.txt
from __future__ import print_function
from dateutil.parser import parse
import re
import pandas as pd

PATTERN = re.compile('\[(.*?)\] /scholia(?:/(.*?)/)?')

filename = "stats.txt"
data = []
for line_no, line in enumerate(open(filename)):
    try:
        datetime_string, aspect = PATTERN.findall(line)[0]
    except: 
        continue
    datetime = parse(datetime_string)
    data.append((datetime, aspect, 1))
    if not line_no % 10000:
        print(line_no)

df = pd.DataFrame(data, columns=['datetime', 'aspect', 'count'])
yearmonth = [datetime.strftime('%Y-%m') for datetime in df['datetime']]
df['yearmonth'] = yearmonth

print(df.groupby('yearmonth').sum())

aggregated = df.groupby('yearmonth').sum()
from pylab import *

aggregated.plot(grid=True, xticks=range(len(aggregated.index)), logy=True, linewidth=5)
xlabel('Year month')
title('Scholia page views')
gca().set_xticklabels(aggregated.index, rotation = 45, ha="right")
show()

Updates for wmflabs

Login to login.tools.wmflabs.org

become scholia
webservice --backend=kubernetes python3.7 shell
source ~/www/python/venv/bin/activate
cd ~/www/python/scholia ; git pull
pip install -rrequirements.txt
logout
webservice --backend=kubernetes -m 1G python3.7 restart
tail -f ~/uwsgi.log  # look at web log
Clone this wiki locally