baseline_2

    main(chosen_model_no=2,
         use_spacy=False,
         use_soft_cosine_similarity=False,
         num_topics=None,
         no_below=5,
         no_above=0.5,
         normalize_vectors=False)

lsi_tf_idf Loading tokens 2019-03-24 23:41:02,996 : INFO : adding document #0 to Dictionary(0 unique tokens: []) 2019-03-24 23:41:09,882 : INFO : adding document #10000 to Dictionary(119578 unique tokens: ['-PRON-', 'action', 'affect', 'ally', 'an']...) 2019-03-24 23:41:13,599 : INFO : adding document #20000 to Dictionary(192872 unique tokens: ['-PRON-', 'action', 'affect', 'ally', 'an']...) 2019-03-24 23:41:16,680 : INFO : adding document #30000 to Dictionary(250954 unique tokens: ['-PRON-', 'action', 'affect', 'ally', 'an']...) 2019-03-24 23:41:16,868 : INFO : built Dictionary(255118 unique tokens: ['-PRON-', 'action', 'affect', 'ally', 'an']...) from 30885 documents (total 7885592 corpus positions) 255118 2019-03-24 23:41:17,312 : INFO : discarding 231029 tokens: [('-PRON-', 30238), ('an', 16990), ('game', 23546), ('in', 27568), ('of', 29090), ('the', 30318), ('this', 15532), ('to', 29821), ('with', 25261), ('a', 29432)]... 2019-03-24 23:41:17,313 : INFO : keeping 24089 tokens which were in no less than 5 and no more than 15442 (=50.0%) documents 2019-03-24 23:41:17,427 : INFO : resulting dictionary: Dictionary(24089 unique tokens: ['action', 'affect', 'ally', 'base', 'brand']...) 24089 2019-03-24 23:41:23,215 : INFO : collecting document frequencies 2019-03-24 23:41:23,215 : INFO : PROGRESS: processing document #0 2019-03-24 23:41:23,518 : INFO : PROGRESS: processing document #10000 2019-03-24 23:41:23,801 : INFO : PROGRESS: processing document #20000 2019-03-24 23:41:24,058 : INFO : PROGRESS: processing document #30000 2019-03-24 23:41:24,079 : INFO : calculating IDF weights for 30885 documents and 24088 features (3234325 matrix non-zeros) 2019-03-24 23:41:24,169 : INFO : using serial LSI version on this node 2019-03-24 23:41:24,169 : INFO : updating model with new documents Corpus as Tf-Idf Latent Semantic Indexing (LSI/LSA) 2019-03-24 23:41:37,862 : INFO : preparing a new chunk of documents 2019-03-24 23:41:38,866 : INFO : using 100 extra samples and 2 power iterations 2019-03-24 23:41:38,867 : INFO : 1st phase: constructing (24089, 300) action matrix 2019-03-24 23:41:40,332 : INFO : orthonormalizing (24089, 300) action matrix 2019-03-24 23:41:48,891 : INFO : 2nd phase: running dense svd on (300, 20000) matrix 2019-03-24 23:41:51,951 : INFO : computing the final decomposition 2019-03-24 23:41:51,961 : INFO : keeping 200 factors (discarding 15.271% of energy spectrum) 2019-03-24 23:41:52,330 : INFO : processed documents up to #20000 2019-03-24 23:41:52,351 : INFO : topic #0(2482.876): 0.120*"player" + 0.102*"or" + 0.097*"new" + 0.095*"one" + 0.092*"first" + 0.087*"world" + 0.085*"mode" + 0.079*"two" + 0.077*"use" + 0.077*"battle" 2019-03-24 23:41:52,352 : INFO : topic #1(1026.409): 0.293*"miner" + 0.242*"Miner" + 0.238*"jewel" + 0.233*"miners" + 0.145*"lmb" + 0.142*"press" + 0.138*"tnt" + 0.134*"button" + 0.118*"LMB" + -0.116*"ship" 2019-03-24 23:41:52,353 : INFO : topic #2(972.048): -0.243*"video" + 0.189*"miner" + 0.156*"jewel" + 0.153*"Miner" + 0.147*"miners" + -0.147*"vr" + -0.128*"tool" + -0.121*"car" + -0.118*"photo" + -0.103*"track" 2019-03-24 23:41:52,355 : INFO : topic #3(964.271): 0.235*"ship" + 0.174*"unit" + -0.157*"puzzle" + 0.122*"war" + -0.117*"adventure" + 0.112*"map" + 0.108*"space" + -0.108*"story" + -0.098*"dungeon" + 0.097*"battle" 2019-03-24 23:41:52,356 : INFO : topic #4(879.265): 0.402*"ship" + -0.348*"card" + 0.239*"space" + 0.186*"planet" + -0.137*"solitaire" + 0.122*"station" + -0.120*"team" + 0.106*"galaxy" + -0.102*"online" + -0.100*"deck" 2019-03-24 23:41:58,753 : INFO : preparing a new chunk of documents 2019-03-24 23:41:59,132 : INFO : using 100 extra samples and 2 power iterations 2019-03-24 23:41:59,133 : INFO : 1st phase: constructing (24089, 300) action matrix 2019-03-24 23:41:59,748 : INFO : orthonormalizing (24089, 300) action matrix 2019-03-24 23:42:05,802 : INFO : 2nd phase: running dense svd on (300, 10885) matrix 2019-03-24 23:42:07,275 : INFO : computing the final decomposition 2019-03-24 23:42:07,275 : INFO : keeping 200 factors (discarding 16.332% of energy spectrum) 2019-03-24 23:42:07,543 : INFO : merging projections: (24089, 200) + (24089, 200) 2019-03-24 23:42:08,856 : INFO : keeping 200 factors (discarding 14.281% of energy spectrum) 2019-03-24 23:42:09,234 : INFO : processed documents up to #30885 2019-03-24 23:42:09,236 : INFO : topic #0(2903.487): 0.117*"player" + 0.101*"or" + 0.098*"one" + 0.093*"first" + 0.089*"new" + 0.087*"world" + 0.080*"mode" + 0.080*"two" + 0.078*"enemy" + 0.077*"use" 2019-03-24 23:42:09,237 : INFO : topic #1(1144.736): 0.224*"ship" + -0.191*"video" + 0.164*"battle" + 0.145*"unit" + 0.142*"war" + 0.129*"enemy" + -0.128*"puzzle" + -0.125*"vr" + 0.121*"combat" + 0.109*"weapon" 2019-03-24 23:42:09,237 : INFO : topic #2(1120.677): -0.153*"video" + -0.138*"ship" + -0.124*"car" + 0.122*"puzzle" + 0.115*"adventure" + 0.115*"story" + 0.110*"dungeon" + -0.097*"vr" + -0.095*"racing" + 0.091*"find" 2019-03-24 23:42:09,238 : INFO : topic #3(1037.708): -0.299*"miner" + -0.245*"Miner" + -0.242*"jewel" + -0.236*"miners" + -0.160*"press" + -0.159*"lmb" + -0.158*"button" + -0.149*"two" + 0.144*"video" + -0.140*"tnt" 2019-03-24 23:42:09,239 : INFO : topic #4(1000.353): -0.402*"ship" + 0.333*"card" + -0.244*"space" + -0.190*"planet" + -0.129*"station" + 0.119*"team" + 0.110*"deck" + 0.108*"player" + -0.106*"crew" + 0.103*"mode" 2019-03-24 23:42:09,316 : INFO : creating matrix with 30885 documents and 24089 features

Query appID: 570 (Dota 2)