Metrics

MESUR metrics: counts and networks.

MESUR investigates an array of possible impact metrics that includes not only frequency-based metrics (citation and hit counts), but also network-based metrics such as those employed in social network analysis and web search engines, e.g. Google’s PageRank uses the web hyperlink structure to rank web pages. To rank journals according to network-based metrics of impact, we need journal networks. Journal networks can be created by connecting individual journals on the basis of a chosen relationship.

Citation networks.

Doing so is relatively straightforward for a citation relationship as citation databases such as Thomson Scientific's Journal Citation Records list the number of citations that point from one journal to another. Each row in the list represents a connection between a given pair of journals, and the number of citations indicates the strength of their connection. A journal citation network results when all connections are taken into account. This approach has been extensively applied in other efforts to map science on the basis of journal citation data.

Usage networks.

Usage networks are created differently. Usage data is not expressed as a list of journal-to-journal connections, but as a flat, time-sequential list of article-level usage events from which journal connections must be derived. MESUR determines these journal connections using a process that is commonly employed by online recommender services such as Amazon.com and Netflix (i.e.~"you may also be interested in these items"). The assumption at the basis of these systems is that the degree to which any pair of items is related is a function of the frequency by which users jointly purchase, download, or access them. This relationship is known as usage co-occurrence, and it is used to create MESUR's journal usage networks. However, since the MESUR usage data doesn't identify individual users, usage co-occurrence was reformulated in terms of sessions, indicated by anonymized session identifiers: the degree of relationship between any pair of journals is a function of the frequency by which they are jointly accessed within user sessions. The figure above illustrates this process. Within a usage data set, usage events are grouped according to the session in which they occur. This allows determining how frequently a given pair of journals is accessed within the same session. This frequency determines the strength of the connection between this particular pair of journals. The connections thus extracted for each pair of journals can then be combined to form a journal usage network.

For the usage and citation networks in the MESUR databases, we have at this point calculated a preliminary set of different “impact” metrics as a proof of principle. We are expanding this initial set in our research program.

Some metrics are network-based, others are based on frequency. This is the list of names for the metrics we calculated organized by the general type of metric. Each metric is preceded by an abbreviation we commonly use in publication. Each metric, except the Thomson Scientific Impact Factor, was calculated for both MESUR’s usage and citation data, leading to a total of 23 usage-based metric, 23 citation-based metrics and the Thomson Scientific Impact Factor.

Degree ~ popularity indicated by number of links/connections:

1) ID: In-degree (Number of links pointing to journal)

2) WID: Weighted In-Degree (Sum of link weights pointing to journal)

3) IE: In-Degree Entropy (Entropy of link weight distribution pointing to journal)

4) OD: Out-degree (Number of links pointing from journal)

5) WOD: Weighted Out-degree (Sum of link weights pointing from journal)

6) OE: Out-degree entropy (Entropy of link weight distribution pointing from journal)

7*) IF: Journal Impact Factor (Thomson Scientific’s Impact Factor)

Shortest Path ~ network distance and “power” positions:

8) UBW: Unweighted Betweenness centrality

Frequency by which journal sits on shortest path between any pair of journal,

shortest path calculation ignore link weights

9) WBW: Weighted Betweenness centrality

Frequency by which journal sits on shortest path between any pair of journal,

shortest path calculation takes into account link weights

10) UBW-UN: Unweighted Betweenness centrality un-normalized

Same as above, but frequency un-normalized by size of connected component

11) WBW-UN: Weighted Betweenness centrality un-normalized

Same as above, but shortest path calculation takes into account link weights

12) UCL: Unweighted Closeness centrality:

Average length of shortest path between journal and all other nodes,

calculation of shortest path does not take into account link weights

13) WCL: Weighted Closeness centrality

Same as above but shortest path calculation takes into account link weights

14) UCL-UN: Unweighted Closeness Un-normalized

Same as above, but un-normalized by size of connected component

15) WCL-UN: Weighted Closeness Un-normalized

Same as above but shortest path calculation takes into account link weights

16) UNM: Unweighted Newman’s “load”

Newman’s version of betweenness, also referred to as load. See Newman (2005), A measure of betweenness centrality based on random walk. Social Networks, 27(1), 39-54.)

Shortest paths do not take into account link weights.

17) WNM: Weighted Newman’s “load”

Same as above but link weights are taken into account

18) UNM-UN: Unweighted Newman’s “load” Un-normalized

Same as above, but un-normalized by size of connected component

19) WNM-UN: Weighted Newman’s “load” Un-normalized

Same as above, but un-normalized by size of connected component

Random Walk ~ prestige given by random walk simulations on network:

20) PR: Particle Swarm’s PageRank

calculated by Marko A. Rodriguez’s Particle Swarm method