arxiv: v1 [cs.dl] 22 Dec 2016

Transcription

1 ScieceWISE: Topic Modelig over Scietific Literature Networks arxiv: v1 [cs.dl] 22 Dec 2016 A. Magalich, V. Gemmetto, D. Garlaschelli, A. Boyarsky Uiversity of Leide, The Netherlads {magalich, palchykov, gemmetto, garlaschelli, A. Martii, A. Cardillo, A. Costati, O. Ruchayskiy, P. De Los Rios, K. Aberer École Polytechique Fédérale de Lausae, Switzerlad {adrea.martii, alessio.cardillo, alex.costati, oleg.ruchayskiy, paolo.delosrios, A. Lutov, M. Khayati, P. Cudré-Mauroux Uiversity of Fribourg, Switzerlad {artem, mkhayati, V. Palchykov Uiversity of Leide, The Netherlads ad Istitute for Codesed Matter Physics, Liviv, Ukraie ABSTRACT We provide a up-to-date view o the kowledge maagemet system ScieceWISE (SW) ad address issues related to the automatic assigmet of articles to research topics. So far, SW has bee prove to be a effective platform for maagig large volumes of techical articles by meas of otological cocept-based browsig. However, as the publicatio of research articles accelerates, the expressivity ad the richess of the SW otology turs ito a double-edged sword: a more fie-graied characterizatio of articles is possible, but at the cost of itroducig more spurious relatios amog them. I this cotext, the challege of cotiuously recommedig relevat articles to users lies i tacklig a etwork partitioig problem, where odes represet articles ad co-occurrig cocepts create edges betwee them. I this paper, we discuss the three research directios we have take for solvig this issue: i) the idetificatio of geeric cocepts to reiforce iter-article similarities; ii) the adoptio of a bipartite etwork represetatio to improve scalability; iii) the desig of a clusterig algorithm to idetify cocepts for cross-discipliary articles ad obtai fie-graied topics for all articles. 1. INTRODUCTION The past several decades have see a sizable growth of the global research commuity, as well as a acceleratio of the scietific publishig workflow. This has led i tur to a drastic icrease of the umber of published works [6, 26], a proliferatio that has carried with it may techological challeges. Oe such challege is the idetificatio ad discrimiatio of itra- vs. crossdiscipliary scietific productio, i order to ease ad ehace bibliographic search. The ability to make such a distictio is see as depedet o the so-called coceptualizig of cotet [25, 27], achievable through the maiteace of otologies of cocepts referred to i specialized literature. The huge amout of available articles, however, requires automatig the costructio of such otological substrates as much as possible, but without compromisig quality with respect to humaexpert relevace judgmets. The ScieceWISE platform 1 (heceforth idicated as SW) aims at facilitatig the ecessary type of humacomputer iteractio to reach the above objective. It provides a iteractive eviromet i support of the scietific commuity, where expert users ca cotiuously cotribute to the maiteace of a data-drive domai otology through routie literature review activities. I doig this, they also give the system implicit feedback o cotet they cosider relevat, which is subsequetly used to provide better recommedatios of ew articles to bookmark ad review. I this paper, we first provide a updated view of the SW platform ad its mechaism for extractig cocepts from scietific articles. We the look at aspects related to establishig a coheret correspodece betwee articles ad broad topics of research. I particular, we will describe how the accurate selectio of relevat cocepts (Sec. 3), the choice of suitable represetatio schemes (Sec. 4) ad accoutig for overlappig 1

2 commuity structure (Sec. 5) are all viable solutios to tackle the topic modelig issue. Fially, i Sec. 6, we report o lessos leared ad sketch future developmet possibilities. 2. PLATFORM OVERVIEW Orgaizig scietific literature i a systematic way calls for effective ad efficiet methods to costructig huma-browsable taxoomies or otologies that are field-specific, kept up-to-date ad iterliked to the cotets of the documets they mea to represet. The extractio of relevat cocepts from ew publicatios thus costitutes the corerstoe of SW. Curretly, the platform s primary source of literature is the arxiv.org e-prit repository of physical scieces articles [11], with which it sychroizes daily. Ay ewly published article is automatically aalyzed for its most relevat cocepts as well as ovel termiology [21, 6]. The article is the made available to the system s users for ispectio ad cocept taggig. Through this process of bookmarkig, users cotribute expert iput o the most relevat cocepts of the article, possibly also specifyig ew cocepts ad sematic relatios amog them i the process, such as specializatio or similarity (Fig. 1). Figure 1. Otological represetatio of the cocept Dark Matter. The etry icludes iformatio about categorizatio, related cocepts, defiitios ad other resources. Upo bookmarkig a article, the user is preseted with three complemetary optios for selectig potetial tags (Fig. 2): choose existig otological cocepts foud i the article; choose automatically extracted keyphrases, that are ot curretly part of the otology; maually add ay missig but relevat terms. The user may choose ay combiatio of the above optios to compile a set of terms that best represets his/her iterest i the article i questio. For coveiece, the list of otological cocepts is raked by a ehaced TF-IDF measure [21], while the list of keyphrases is geerated ad raked usig the KPEX algorithm [6]. New user cotributios (cocepts ad/or relatios) are periodically validated by a group of system curators (power-users) before beig permaetly adopted as updates to the curret otology. Figure 2. Article bookmarkig iterface. The colum Foud cocepts cotais otological etries, Possible cocepts ovel extracted keyphrases, ad Chose cocepts the user s persoal choice. As a additioal feature, bookmark collectios may be aalyzed to provide each user with persoalized article recommedatios. Sice scietific literature is geerally quite specific, the co-occurrece of cocepts withi the same article ofte suggests a thematic relatio betwee them, whilst the presece of the same cocept i differet articles is a buildig block to gauge the relatio betwee differet articles. This feature favors the use of SW as a persoalized sematic filter over the stream of ew, icomig articles. It is at this poit, however, that ew methods for derivig meaigful relatios betwee articles are eeded. The iformatio extracted from SW ca, i tur, be represeted as a set of elemets belogig to two distict classes: articles ad cocepts, which ca be ecoded as a complex etwork [5, 17]. A etwork represetatio ca be very effective i highlightig importat relatios. Whe the uderlyig otology becomes too rich, however, the probability that two articles share oe cocept by chace becomes o-egligible. This pheomeo is resposible for the emergece of may spurious iteractios that raise the desity of coectios ad cosequetly makes the iformatio extractio process harder. We had to devise a umber of ew solutios to work aroud this etwork desity problem i SW. The followig sectios showcase three successful but complemetary ways to address this issue, ad that are applicable at differet stages of the processig pipelie a better selectio of relevat cocepts, a more effective etwork represetatio ad a more meaigful partitioig of the etwork. 3. ENTROPIC IDENTIFICATION OF RELEVANT CONCEPTS Commo or geeric cocepts preset i most articles (such as model or star i physics) create may spurious iteractios i a etwork represetatio. The resultig structure is thus very dese, resemblig a complete graph where all odes (articles) have a ear-direct lik to each other. Several methods have already bee proposed

3 to tackle the problem of droppig some of the liks i dese etworks to make the etworks sparser [24, 22] but they operate i a ex-post way, validatig coectios i a resultig etwork based o their weights. I our cotext, the challege is istead to reduce the umber of coectios ad the effect of spurious similarity o the weights a priori, by filterig away the commo (or geeric) cocepts, which do ot provide ay valuable classificatio iformatio. This is already achieved i SW by a crowdsourced cocept classificatio, i.e. through the assigmet of a geeric cocept flag. Yet, the cocepts that may be cosidered as geeric i oe disciplie may be quite specific i aother. This makes the problem of idetifyig commo cocepts strogly depedet o the cosidered collectio of papers ad their topics. Hece, the problem requires a solutio that does ot rely o huma-made iput, but rather supplemets it. Our solutio to this problem cosists i makig use of etropy to select relevat cocepts. Etropy has previously bee used i sematic aalysis studies to classify documets [12, 3] based o the frequecy of words i a text i.e. term frequecy T F [30, 8, 9, 10]. Specifically, we make use of Shao s etropy to quatify the amout of iformatio carried out by cocepts. For each cocept c, we compute two differet etropies: oe, Sc, associated to the observed T F probability distributio, ad the other, Scmax, associated to a probability distributio derived from the maximum etropy priciple ad costraied by the first momet ad log-momet of the observed T F distributio [3, 1, 29]. The maximum etropy value acts as a upper boud for the observed etropy, allowig to determie if the observed value is big or small with respect to a quatity computed for the same cocept. Hece, the compariso of the two etropy values (Fig. 3) costitutes the core of the methodology we used to filter the similarity etwork. More specifically, i Fig. 3 we cosider the distributio of poits i the Sc vs Scmax space for the cocepts extracted from all the articles submitted durig the year 2013 to the arxiv i physics subject classes as pricipal category (hereafter arxivphys2013pc), which comprises early 30k articles. A glace at the diagram reveals some itriguig features. For istace, the vast majority of had-labeled geeric cocepts (black dots) sit alog the Sc = Scmax lie, suggestig that geeric cocepts ted to maximize their etropy ad, cosequetly, carry little iformatio. However, a more careful aalysis (displayed i the iset) hits at the presece of may more cocepts whose observed etropy is close to the maximum. This paves the way for the establishmet of a parallel betwee geeric cocepts ad those with a observed etropy closer tha a give threshold p to their maximal oe Scmax. Figure 3. Relatio betwee the etropy Sc of a cocept c ad Scmax, for the collectio arxivphys2013pc. The backgroud colors delimit the regios where the distace betwee a poit ad the lie Sc = Scmax is less tha a give percetile p. Black dots idetify the cocepts marked as geeric by experts. The secod iterestig feature is the presece of a few groups of commo cocepts located away from the boudary. These outliers have bee marked by users who might correctly cosider them as commo i their ow sub-disciplie, but which might represet oly a small fractio of the whole dataset. O the etwork desity side, the pruig of basic cocepts allows to drop sigificatly the lik desity. As a example, by removig cocepts fallig withi the 10th percetile, we are able to decrease lik desity by 20%. Such a drastic reductio leads to a clearer ad more specific commuity structure ad topic classificatio. Furthermore, the automatic idetificatio of commo cocepts ca thus be used to sigal potetially wrogly-defied geeric cocepts. 4. ALTERNATIVE GRAPH REPRESENTATIONS The set pf articles ad cocepts used by SW has thus far bee represeted as a uipartite (up) etwork, where oly articles are odes. Alteratively, the system may be represeted as a bipartite (bp) etwork, where its articles ad cocepts ca be directly mapped to two types of odes. Liks (edges) oly coect cross-type odes articles to their respective cocepts. Such a alterative represetatio elimiates the desity problem sice the umber of liks i a bipartite etwork grows liearly with the umber of articles,, while it grows as O(2 ) i the uipartite case (see Fig. 4(a)). Cosequetly, the bipartite approach sigificatly reduces the computatioal resources eeded to cluster big collectios without loss of quality, as show below. Besides the arxivphys2013pc dataset, we also

4 Number of liks a up 10 3 bp c 0.9 R (stability) up bp t R (similarity) b up bp 1.0 d Figure 4. (a) Number of liks as a fuctio of the etwork size. (b) Ru-time required to compute oe sigle realizatio of the Louvai algorithm as a fuctio of ; results refer to its pytho-igraph implemetatio. (c) Stability of the outcome of the stochastic algorithm for the two differet etwork represetatios. (d) Similarity betwee the up ad bp partitios. cosider 1084 of its subsets of differet sizes accoutig for sigle arxiv categories (7k 11k articles); sub-categories (280 3k articles) ad smaller groups of papers (with at least 20 odes) that are rather focused o specific topics i their ow disciplies. All these subsets have bee selected uiformly by recursively applyig modularity optimizatio methods [4] to the up represetatio of the etire collectio ad the resultig clusters/sub-sets [20]. We ivestigated ru-time depedece o the size of the dataset for the up ad bp represetatios usig the Louvai clusterig algorithm [4], which aims at maximizig the modularity fuctio [18]. Origially, the modularity fuctio had bee desiged for uipartite etworks. To deal with bipartite etworks, Barber s geeralizatio of modularity [2], which adjusts the uderlyig ull model, was employed. However, our prelimiary examiatio shows that the choice of the ull model is ot crucial, as it barely affects the commuity detectio results. This motivates us to use the same implemetatio i order to make a cosistet compariso betwee the resources required to cluster articles i the two represetatios. The ru-time results are showcased i Fig. 4(b), where the average time to cluster a collectio of a give size is supplemeted by the stadard deviatio. I both represetatios, the ru-time t scales as a power-law fuctio of the size as t α. However, the scalig is close to quadratic for the up represetatio (α 2.20), but about liear for the bp oe (α 1.15). The differece is likely due to the aforemetioed discrepacy i the umber of liks betwee the two represetatios, which follows roughly the same tred. For small datasets (below 100 articles), a uipartite approach is more coveiet, while for bigger collectios the bipartite represetatio prevails. Noticeably, there is a 100 times differece i the ru-time for collectios of = 10 4 articles. Sice the heuristic used be the Louvai algorithm has a stochastic compoet, the results of differet rus may, i geeral, differ. Thus, it is crucial to uderstad how stable the outcomes of differet rus are: is it ecessary to cosider a optimal partitio over may rus or is it eough to take just oe ru? To aswer this questio, we first idetify 100 partitios for each dataset ad each etwork represetatio. The, we cosider all possible pairs of these outcomes ad perform pairwise comparisos usig Rad Idex R [23]. The behaviors of R, show i Fig. 4(c), idicates that a sigle ru of Louvai suffices for ay up etwork ad for small bp etworks. Although stability is a fudametal requiremet, it is also importat to establish how similar two partitios are withi differet represetatios. We therefore compared each of the 100 partitios of up represetatios with each partitio obtaied from the bp represetatio for every sigle cosidered dataset. The results i terms of R are show i Fig. 4(d): sigificat discrepacies for small datasets have bee observed, while the two represetatios lead to similar outcomes for larger collectios. Comparig these results to the stability results (Fig. 4(c)), we coclude that the similarity betwee up ad bp partitios is closely approachig the (smallest) stability idex of the resultig partitios for all datasets above 100 articles. This meas that the clusterig obtaied usig the two alterative approaches are similar, though this similarity is limited by the stability restrictios of each approach. To summarize, our resource ad quality performace tests have highlighted uder which coditios bp ad up approaches may be used as alterative ways to cluster article collectios with comparable outcomes. There is a clear trade-off betwee computatioal resources ad stability: smaller datasets should be clustered usig the up represetatio (good stability, low resource cosumptio) while larger datasets should employ bp clusterig despite less stable results. 5. OVERLAPPING COMMUNITIES I the above sectios, we cosidered the ooverlappig clusterig of articles where each mauscript belogs to a sigle topic. However, there are may cross-discipliary articles whose cotets do ot belog to a sigle well-defied topic. I this sectio, we therefore cosider overlappig clusterig [15], ad istead of maximizig the usual modularity fuctio, we maximize its exteded versio [19].

5 I this cotext, we propose a greedy optimizatio algorithm, HiReCS, ispired by the Louvai routie. At each sigle iteratio it merges iitially ugrouped odes ito commuities usig maximal a Mutual Modularity Gai (MMG) criterio. More specifically, for each ode #i i the etwork, we idetify a set of eighbors such that the pair-wise merge of #i with each of these leads to a maximal gai i modularity. The, two odes #i ad #j are merged ito a cluster if such a move is optimal for both of them. A overlap appears whe for a give ode #i several eighbors satisfy the MMG criterio equally. To represet the overlap i the shared ode #i, we replace #i by a set of replica odes #ik, oe for each of the K overlappig clusters C k. To esure the coservatio of the weights of the liks icidet to a shared ode, we equally distribute them across all the replicas as follows: k {1,..., K}; w ik,j = w i,j K, where w ik,j stads for the weight of the lik betwee #ik ad #j. Ulike the Louvai algorithm, HiReCS groups the odes ito fie-graied clusters i a determiistic way. The clusterig itself is a agglomerative hierarchical method, where each clusterig iteratio builds a subsequet level of the hierarchical commuity structure. At each iteratio, the o-clustered odes are propagated to the ext iteratio where they are treated as iput odes, as well as the previously formed clusters. The executio termiates whe a ew iteratio does ot produce ay ew cluster. To assess the quality of the results, our algorithm was applied to sythetic etworks with overlappig clusters geerated by the LFR bechmark [14]. We compare HiReCS agaist state-of-the-art algorithms, icludig Louvai [4], Gaxis [28], Oslom2 [15], SCP [13] ad a radom commuities geerator (Rcoms). The latter takes the umber ad size of clusters from the groudtruth ad assigs coected odes radomly to clusters. To assess the similarity betwee the resultig clusters ad the groud-truth commuities, we use the Normalized Mutual Iformatio (NMI) for overlappig clusters [7], which is fully compatible with the stadard NMI [16]. The NMI values vary i the rage [0, 1], where a value of 1 sigifies that a clusterig is idetical to the groud-truth. The results show i Fig. 5 show that HiReCS outperforms state-of-the-art solutios, icludig both parameter-free ad parameterized oes o sparse etworks, due to the fie-graied structure of the clusters resultig from the clusterig. Fie-graied cluster idetificatio is importat to trace ad iterrelate topics from the most commo oes NMI HIRECS SCP Louvai Oslom # odes (x 10 3 ) Gaxis Rcoms Figure 5. Compariso of NMI values achieved by differet clusterig algorithms, as a fuctio of the umber of odes i the bechmark etwork. Results are averaged over eight etwork istaces. to the most specific oes, as well as to idetify topics i cross-discipliary articles. Those experimetal results show that our algorithm grasps the commuity structure better tha its competitors i a parameter-free way ad allows to idetify multiple topics for a give paper. Additioally, HiReCS achieves a liear space complexity ad ear-liear rutime complexity o sparse etworks with respect to the umber of liks. 6. CONCLUSIONS 6.1 Summary I this paper, we showcased the ovel methods that SW employs to address the problem of sematic orgaizatio ad cotets retrieval o large collectio of scietific papers. Sice its lauch i 2009, the system has reached several milestoes, evolvig from the whim of a few scietists to a well-established platform used by hudreds of users o a daily basis. Based o the experiece accumulated by developpig ad usig SW over the years, we are able to pipoit some importat lessos leared so far, ad to use them as departure poits for the further developmets of the system. I this article, we highlighted three key issues related to the assigmet of topics to documet etworks. More specifically, we showed how: The automatic selectio of relevat cocepts based o etropy favors the emergece of stroglyorgaized structures also at the level of subtopics ad, more importatly, does ot rely o huma evaluatio/validatio. The bipartite represetatio of the system ca be used to overcome the limitatios imposed by the desity of the uipartite graph. Moreover, the stability of the results, albeit beig iferior to the uipartite case, remais still high.

6 The HiReCS algorithm, which supports overlappig commuities, ca be applied to improve both the classificatio of cross-discipliary articles ad the idetificatio of fie-graied topics for the articles. 6.2 Future Work A umber of importat challeges lay ahead. We are curretly workig o overtakig the curret limitatios of the methodologies preseted i this mauscript, for example, by combiig together two or more approaches. Aother challege relates to the applicatio of the isight extracted from physics-related articles (for which a well-established otology exists) to other disciplies such as biomedicie or evirometal scieces, ad to o-scietific media altogether, where otologies are scarce or eve ofte missig. I a wider perspective, SW ca be see as a ideal ecosystem to test various search, clusterig ad data maagemet techologies, ragig from commuity detectio ad topic modelig, etity likig or kowledge extractio. Ackowledgmets This work was supported by SNSF project No (Crowdsourced coceptualisatio of complex scietific kowledge ad discovery of discoveries) ad by the EU project MULTIPLEX (cotract ). Refereces [1] S. K. Baek, S. Berhardsso, ad P. Mihage. Zipf s law uzipped. New Joural of Physics, 13:043004, [2] M. J. Barber. Modularity ad commuity detectio i bipartite etworks. Phys. Rev. E, 76:066102, [3] A. L. Berger, V. J. Della Pietra, ad S. A. Della Pietra. A maximum etropy approach to atural laguage processig. Comput. Liguist., 22(1):39 71, [4] V. Blodel, J.-L. Guillaume, R. Lambiotte, ad E. Lefebvre. Fast ufoldig of commuities i large etworks. J. Stat. Mech. Theor. Exp., 2008(10):P10008, [5] S. Boccaletti, V. Latora, Y. Moreo, M. Chavez, ad D.-U. Hwag. Complex etworks: Structure ad dyamics. Physics Reports, 424: , [6] A. Costati. Automatic structure ad keyphrase aalysis of scietific publicatios. PhD thesis, [7] A. V. Esquivel ad M. Rosvall. Comparig etwork covers usig mutual iformatio. CoRR, abs/ , [8] R. Ferrer i Cacho ad R. V. Solé. Least Effort ad the Origis of Scalig i the Huma Laguage. Proceedigs of the Natioal Academy of Sciece (USA), 100: , [9] F. Fot-Clos, G. Boleda, ad A. Corral. A scalig law beyod zipf s law ad its relatio to heaps law. New Joural of Physics, 15(9):093033, [10] M. Gerlach ad E. G. Altma. Scalig laws ad fluctuatios i the statistics of word frequecies. New Joural of Physics, 16:113010, [11] P. Gisparg. ArXiv at 20. Nature, 476(7359):145 7, [12] A. Hotho, A. Nürberger, ad G. Paaß. A brief survey of text miig. LDV Forum - GLDV Joural for Computatioal Liguistics ad Laguage Techology, [13] J. M. Kumpula, M. Kivelä, K. Kaski, ad J. Saramäki. Sequetial algorithm for fast clique percolatio. Phys. Rev. E, 78:026109, [14] A. Lacichietti ad S. Fortuato. Bechmarks for testig commuity detectio algorithms o directed ad weighted graphs with overlappig commuities. Phys. Rev. E, 80:016118, Jul [15] A. Lacichietti, F. Radicchi, J. J. Ramasco, ad S. Fortuato. Fidig statistically sigificat commuities i etworks. PLoS ONE, 6:e18961, [16] C. D. Maig, P. Raghava, H. Schütze, et al. Itroductio to iformatio retrieval, volume 1. Cambridge uiversity press Cambridge, [17] M. E. J. Newma. Networks. Oxford Uiversity Press, [18] M. E. J. Newma ad M. Girva. Fidig ad evaluatig commuity structure i etworks. Phys. Rev. E, 69(2):026113, [19] V. Nicosia, G. Magioi, V. Carchiolo, ad M. Malgeri. Extedig the defiitio of modularity to directed graphs with overlappig commuities. Joural of Statistical Mechaics: Theory ad Experimet, 2009(03):P03024, [20] V. Palchykov, V. Gemmetto, A. Boyarsky, ad D. Garlaschelli. Groud truth? cocept-based commuities versus the exteral classificatio of physics mauscripts. arxiv: , [21] R. Prokofyev, A. Boyarsky, O. Ruchayskiy, K. Aberer, G. Demartii, ad P. Cudré-Mauroux. Tag recommedatio for largescale otology-based iformatio systems. I The Sematic Web ISWC 2012, pages Spriger, [22] F. Radicchi, J. J. Ramasco, ad S. Fortuato. Iformatio filterig i complex weighted etworks. Phys. Rev. E, 83:046101, [23] W. M. Rad. Objective criteria for the evaluatio of clusterig methods. Joural of the America Statistical Associatio, 336(66): , [24] M. A. Serrao, M. Boguñá, ad A. Vespigai. Extractig the multiscale backboe of complex weighted etworks. Proceedigs of the Natioal Academy of Scieces (USA), 106: , [25] N. Shibata, Y. Kajikawa, Y. Takeda, ad K. Matsushima. Detectig emergig research frots based o topological measures i citatio etworks of scietific publicatios. Techovatio, 28(11): , [26] R. Va Noorde. Global scietific output doubles every ie years. global-scietific-output-doubles-everyie-years.html, (accessed 15/02/2016). [27] L. Waltma ad N. J. Eck. A ew methodology for costructig a publicatio-level classificatio system of sciece. Joural of the America Society for Iformatio Sciece ad Techology, 63(12): , [28] J. Xie, B. K. Szymaski, ad X. Liu. Slpa: Ucoverig overlappig commuities i social etworks via a speaker-listeer iteractio dyamic process. I ICDMW 11, ICDMW 11, pages IEEE Computer Society, [29] X. Ya ad P. Mihage. Maximum Etropy, Word-Frequecy, Chiese Characters, ad Multiple Meaigs. PLOS ONE, 10:e , [30] G. K. Zipf. Huma behavior ad the priciple of least effort: A itroductio to huma ecology. Laguage, 26: , 1950.