Scientific conferences and their impact metrics are important for the Computer Science community. However, currently, there is no tool that is universally accepted for accessing their citation-based impact metrics, since the tools that provide such information diverge from each other. We claim that the divergence between metrics calculated by different tools is, to a great extent, due to the divergence between the list of papers they consider. In fact, in many cases, existing tools require users to write a query to retrieve the papers of the conference she is interested in knowing the impact metric. This may represent a problem since it may require the user anticipate all the ways that the names of the conference can take on citations of its papers.
To tackle this problem, we developed SHINE, a tool for evaluating impact metrics of Computer Science conferences and events alike. It calculates impact metrics of a conference through the aggregation of citations to each paper that appeared in the conference. Let V be a conference and let Pi be a list of the papers published in V in the year i. Each paper pij in Pi has a number C(pij) of citations obtained from Google Scholar. The number C(pij), used for computing the impact of V (for instance, using the H-index), is obtained by matching each specific paper pij, its title, authors, and year, with the results of a query sent to GS. By doing so, we seek to avoid the inherent ambiguity of querying Google Scholar by venues name or acronym.
Notice that this requires taking the lists of papers of each edition of the conferences of interest from such sources as DBLP, IEEE Explore, ACM DL. Besides these, the Special Commissions of SBC (Brazilian Computer Society) have also indicated other sources.
The first version of SHINE (A Simple H-INdex Estimator) is available at http://shine.icomp.ufam.edu.br
, and it is now frozen. This blog covers the new version, we call Live SHINE.
For this version, we slightly change the acronym to mean now: A S
imple and H
umble Impact In
stimator, to reflect the fact that we want to include other metrics besides H-index.
SHINE is totally non-profit and open to all CS community. Indeed, it has gained popularity among the Brazilian community since its public release about. Importantly, researchers are constantly interacting with our team to point such problems as missing conferences, omissions in lists of papers, etc. SHINE was also adopted by international communities in Italy, Spain and Chile.
SHINE vs. Live SHINE
For interacting with GS, the original SHINE used centralized crawler that harvested the citations data and other meta-data of papers from conferences we have in our database. As we fully observe the operational limits of the service GS provides, the harvesting is slow. By the beginning of the last year, Google Scholar strengthened its police against mass data collection, as we did in SHINE. So, we were unable to update citation data and were forced to freeze SHINE.
Recently, we finished a new piece of software we developed to better comply with policies by Google Scholar in order to update our citations counts. This software is called Live SHINE and runs as a Google Chrome plugin. It looks very much like the original SHINE. However, while the user is using the interface to view a given conference, the plugin sends queries in background to Google Scholar to update the citation counts of the papers of this conference. By doing so, the user “donates” its queries to improve SHINE’s database. Notice that the crawler now runs in a distributed fashion in each client as a form of “crowd crawling”.
Live SHINE runs as a plugin for Google Chrome and it is activated while using Google Scholar. Instead of having the user to type an ad-hoc query to GS, it composes a query on behalf of the user and sends to GS, so to retrieved the data required to update the citation counts of the papers of the conference and provide more reliable impact metrics. Once updated, the citation counts and impact metrics are available to all users.